Factoidd update: major release for Factoshi's accounting script

Previous Updates
Secured
-----------------------------------
ORIGINAL PLEDGES START
-----------------------------------
-DEVELOPMENT

--Develop Automated Filesystem Object Hashing Tool

--Develop Node Monitoring and Alert Tool

--Develop data analytics tools

--Develop SME intellectual property tools

--Establish as an EC reseller

-TESTNET

--Two Testnet nodes

-OTHER

--Commit a hash of known good file state for key security files to Factom at regular intervals

--Publish a monthly blog post

- PLEDGED EFFICIENCY

2 nodes: 50%
1 node: 50%
-----------------------------------
ORIGINAL PLEDGES END
-----------------------------------
-----------------------------------
Relevant links:
- Forum Q+A
- Aggregated pledges
- Application
- Supporting documents
-----------------------------------
-----------------------------------
Secured
Completed

- Data Analytics Tool
A much better way to describe this is as a 'Factom metrics website'. We completed the website in Q3. You can see the finished product here: https://factoshi.io. We plan to launch version 2 next year.

- Node Monitoring and Alert Tool
The first version of this was published a while ago. I continue to make improvements. Recently, I updated it to include a rota, so large teams can manage who is to be alerted according to a weekly timesheet. You can see that change on the Github repo: https://github.com/Factoshi/factom-node-monitord

- Two testnet nodes
We run two nodes on the testnet. One is an authority node. The other is a follower that we use for development and general testing purposes.

- Efficiency
We are operating at 50% efficiency. Our initial pledge was to maintain this for at least 12 months, but there are no plans to lower it after that period.

Queued

- Establish as an EC reseller
This is still on the horizon. It has not been started yet. That is partially because there is currently no demand for EC, and partially because we are busy with other things that we hope will generate that demand.

- File system object hashing tool
Again, this is still on the list of tasks to complete. We still intend to do it, and can develop it in parallel with our current project (more on that below).

- Commit a hash of known good file state for key security files to Factom at regular intervals
We need the file system hashing tool to achieve this. We can start doing that once it has been developed. Hopefully by EOY depending on how well work is progressing on our current large project. It's not a matter of difficulty; it's just a matter of finding time.

Suspended

- SME intellectual property tool
This was going to be the eventual destination of the file system object hashing tool mentioned above. However, we have suspended this pledge indefinitely. We may return to it at a later, but have no firm plans to do so at this time.

The suspension of a core pledge might raise some eyebrows. However, since making our pledges, we have come up with a plan that we think is more commercially viable and is a much more interesting showcase in the power of Factom (more on that below). We chose to do this on the basis that it is likelier to produce better outcomes for both Factom and Factoshi. We are confident in that decision.

Ongoing

- iOS
This is the project that has replaced the intellectual property tool. I don't want to reveal the function of the application yet, though at least two guides are aware of the details. We have wireframed the app, contracted a graphic designer, completed most of the necessary research, and have begun to actively write code.

Launch is currently targeted for Q1 2019. We will have more information closer to that time.

Non-pledge contributions

- Accounting script
We wrote an accounting script to help ANOs manage their accounts. A number of ANOs have adopted it into their workflow. You can see that here: https://github.com/Factoshi/factoid-address-monitord

- Committee participation
I am an active member of the campaign pledge committee.

- Community participation
I am active in community governance decisions. I am also frequently on hand to help people with tech questions.

Summary

We are very pleased with our work so far. We are confident in our course of action, and look forward to sharing more of our work with the community as it becomes available.
Secured
I have just published version 2 of the Factom accounting script. This version is significantly more useful, flexible and maintainable than the previous version. New features include:

- Monitor multiple addresses from one script.
- Output transaction data to only bitcoin.tax, csv, or both.
- Use a combination of different bitcoin.tax accounts for each address.
- Monitor only coinbase transactions, only non-coinbase transactions, or both.
- Use a different base fiat currency for each address.
- Report the TXID for each transaction on bitcoin.tax.
- You no longer need to run walletd.

Please find full instructions for setting it up on the Github repo: https://github.com/Factoshi/factoid-address-monitord

Feel free to reach out to me on Discord if you need any help setting it up.
Secured
Factoshi is moving to a 60% efficiency effective today.

Our website, Factoshi.io, will continue to be maintained and developed. However, health concerns are forcing me to re-evaluate the amount of time I can dedicate to Factoshi and to Factom, which is currently my full time occupation. I believe 60% will bring Factoshi into alignment with other ANOs.

In previous updates I had outlined plans to develop a smart phone application that uses a Factom back end. That will be put on hold until a time that more hours can be invested into the business.

Though this rearrangement is not done willingly, I am pleased to be able to raise Factoshi's efficiency. The last grant round was significantly oversubscribed. Whilst the price has risen recently, we are on course to have even less FCT in the grant pool by the time the next grant round arrives. I consider the grant pool to be a vital resource for protocol development. Generally speaking, I am thoroughly impressed by the quality of the grant proposals we received at the last round, and I am pleased that increasing Factoshi's efficiency will make more funds available to future grant applicants.
Secured
Factoshi is proud to announce a new version of factoshi.io, the Factom community’s central hub for metrics and statistics about the Factom blockchain.

This release includes an entirely new UI and UX to ensure that the tools available to the Factom ecosystem remain competitive with the best that the world of crypto has to offer. Additionally, the website has a number of new features, including:
  • Per block updates - the data you see will always reflect the current state of the blockchain within 10 seconds of the most recent block.
  • Live data - no need to refresh, all new data is streamed live to the site.
  • Redesigned charts - read latest network stats with more clarity.
  • Zoom controls - dig deeper into the data on pre-set time intervals.
  • Simple moving average - set your SMA period so you can get a clear picture of both short- and long-term trajectories.
  • New statistics - see the average TPS for each block over the last 24 hours, average efficiency, and more.
  • Explorer links - all addresses and chain IDs link directly to an explorer, so you can get specific information fast.
  • Grant pool calculator - get an estimate of how many FCT will be in the grant pool on a given date.
Moreover, you now have the opportunity to personalise the site:
  • Dark mode - surf at night without blinding yourself :cool:
  • Default zoom - set the default length of time that the charts cover.
Finally, if you need to grab metrics programmatically, you’re in luck. We have a new API that is friendly to outside users; you can grab all the data on the site with code!
Secured
The recent launch of the Factoshi metrics API has enabled me to contact various price aggregators in order to update their outdated supply information. The following websites have now integrated Factoshi's API.

- CoinMarketCap
- CryptoCompare
- CoinGecko

I am still waiting for response from several others, and I will update this thread if/when I hear back from them. Note that CoinMarketCap has informed me that it could take up to 72 hours for the new data to be reflected on their website.

Thanks to @David Chapman who helped me draft a response to some difficult questions from CoinMarketCap.
Secured
Factoshi increased efficiency at the end of last year from 50 to 60%. At that time, I believed it would bring Factoshi into alignment with other ANOs, as I planned to invest less time in the protocol in order to deal with some pressing issues.

As it turns out, during that period I invested no less time in the protocol. Factoshi has made, and continues to make, significant contributions. As such, Factoshi will be changing its efficiency to 50% in order to come back into alignment with other ANOs.
Secured
I contributed the new FactomEventEmitter class to the FactomJS library. FactomEventEmitter lets you listen for various Factom events without needing to handle polling logic yourself. It simplifies writing applications that react to on-chain events. The following snippet demonstrates the on-chain events you can listen to.

JavaScript:
const { FactomCli, FactomEventEmitter } = require('factom');
const cli = new FactomCli();
// Poll the blockchain every 10s
const emitter = new FactomEventEmitter(cli, { interval: 10000 });
emitter.on('newDirectoryBlock', (directoryBlock) => ...);
emitter.on('newFactoidBlock', (factoidBlock) => ...);
emitter.on('newAdminBlock', (adminBlock) => ...);
emitter.on('newEntryCreditBlock', (entryCreditBlock) => ...);
emitter.on('newChain', (entryBlock) => ...);
// Listen to any transaction involving a given Factoid address
emitter.on('FA29eyMVJaZ2tbGqJ3M49gANaXMXCjgfKcJGe5mx8p4iQFCvFDAC', (transaction) => ...);
// Listen to any new entries in a given chain
emitter.on('4060c0192a421ca121ffff935889ef55a64574a6ef0e69b2b4f8a0ab919b2ca4', (entryBlock) => ...);
https://github.com/PaulBernier/factomjs/blob/master/src/factom-event-emitter.js

Thank you very much to @Paul Bernier for providing thorough code review and feedback.
Secured
Factoshi has been working with Flipside Crypto to integrate Factom metrics into our FCAS score. That work is now complete, and I have been told that our blockchain data will be reflected in our FCAS rating on Coinmarketcap within the next week. You can see our FCAS score here: https://coinmarketcap.com/currencies/factom/#ratings

Factoshi is using data from OpenNode to collect metrics for Flipside Crypto. Thank you to the OpenNode team for providing that high availability endpoint for the community.
Secured
Factoshi has released a new version of our accounting script, which is now called Factoidd.

The new release includes substantial UX and stability improvements. It can be pulled directly from Docker Hub to run in a container, and has default support for OpenNode. The config file is in a more human-friendly YAML format, and will give much improved log output.

https://github.com/Factoshi/factoidd
https://hub.docker.com/r/factoshi/factoidd

Please let me know if you need support to set it up or upgrade. This release is not backwards compatible.
Secured
Factoshi continues to work hard on our primary mission, which is to provide outstanding infrastructure to the Factom network and to build a robust ecosystem that is competitive with high tier public blockchains.

Infrastructure

Factoshi's authority node infrastructure performed as expected during Q3. Server uptime was 100% and factomd uptime equaled that of the Factom network, which experienced a pause during August.

Since being on boarded in Spring 2018, Factoshi has been running its authority servers in line with the specification promised during the campaign. Until now, the specification for Factoshi's authority servers was 8 vCPU/32GB. That spec is far in excess of what factomd has ever used. Even during periods of high network load, factomd has not come close to using those resources; EPS bottlenecks are not currently caused by CPU or memory limitations on authority set nodes.

The unrelenting fall in the price of FCT has compelled a review of that specification due to the potential for cost saving. As a result, Factoshi is reducing authority server specification to 6 vCPU/16GB. This will still leave ample room for growth whilst providing Factoshi welcome reprieve during this period of extremely negative price action. Server performance will be monitored closely, though I consider this to be a low risk change.

Factoshi.io

PegNet metrics were added to Factoshi.io to mark the launch of PegNet in August. This worked was completed in concert with TFA, who contributed the code to support PegNet analytics on the backend. Traffic to the website has doubled since the launch of PegNet, and it is now regularly cited on social media and elsewhere.

Furthermore, the rest of the website received a facelift. Several outstanding bugs were fixed on both the front and back end. The style was also modified slightly to make the website appear more professional.

PegNet

Factoshi's contributions to PegNet fall into 3 rough categories:

1. Porting and maintaining documentation. Work here includes creating and updating the wiki, in addition to helping to add the white paper to GitHub.

2. PegNet Compose. Work was started on PegNet Compose, which will help people get started on mining more easily. Work on PegNet Compose was completed following the end of the quarter.

3. Community support. I have been on standby on the PegNet Discord server to help newcomers troubleshoot issues getting set up.

Accounting script

Factoshi's accounting script received a major upgrade. You can read about that in our previous update.
Secured
Factoshi had a productive Q4. We continue to operate at 50% efficiency whilst contributing to governance and other community endeavours.

1. PegAPI

Factoshi wrote and hosts a PEG price aggregator. The purpose of this price aggregator is to provide a price source to miners on PegNet. PegAPI was instrumental in ensuring that PegNet could fork to a free floating PEG price in early December. The price aggregator is open source and can be found here.

2. Factom Rust Client Contributions

Two PRs have been made to the Factom Rust client. The first of these added multiple utility functions for creating, validating and encoding/decoding addresses. The second addresses a bug in the Admin Entry type. Both PRs are still pending and need some changes due to conflicts that have since arisen.

3. PEG Conversion Guide

I wrote a comprehensive step-by-step guide to converting FCT to PEG. This guide was cited routinely in the period leading up to the December fork.

4. Factoshi.io Growth

Factoshi.io was cited hundreds of times during Q4 as excitement grew over PegNet and FCT burn. During that time, monthly active users rose by more than 200%. Factoshi.io continues to perform its function as an essential resource for community engagement and network transparency.

A note on efficiency and productivity

Recently, several ANOs have lowered (or plan to lower) their efficiency in order to fund continuing development and operation at low FCT prices. Factoshi plans to continue to operate at 50% efficiency. However, in order to manage an increasingly tight budget, productivity will slow for the foreseeable future. This is an inevitable consequence of deeply strained resources. As such, until a time that more resources can be deployed, server revenue will only be used to pay for server costs, factoshi.io hosting and bug fixing, and various essential business expenses.
Secured
1. Factoshi API

Work on the Factoshi stats API is well underway. It coincides with a large refactor of the Factoshi stats daemon to make it easier to add new stats to factoshi.io in the future (such as rich lists). An alpha version should be complete during Q2.

2. Factom.js

I have made several PRs to Factom.js to fix bugs and improve typings:

https://github.com/PaulBernier/factomjs/pull/17
https://github.com/PaulBernier/factomjs/pull/18
https://github.com/PaulBernier/factomjs/pull/19

3. Governance

Factoshi continues to play an active role in governance. I brought forward an informal poll to discover ways to improve the efficiency of the Factom Protocol. I also tried to implement one of those proposals on behalf of ANOs, though the amendment failed at the ratification stage.

Other governance activity includes thorough review and editing of Doc 002 prior to the amendment vote, and general participation and proactive engagement with various governance initiatives.

4. Efficiency

At Factoshi's Q4 2019 update, I mentioned that Factoshi would only be engaging with server maintenance and bug fixing on Factoshi.io in order to reduce costs. As the above development work demonstrates, that did not happen. Continuing to operate at 50% is not sustainable at these price levels. Consequently, Factoshi is dropping to 40% efficiency until a time that the price of FCT can show sustained recovery.
Secured
I am pleased to announce that longstanding community member @WB has joined Factoshi. WB, also known as Vidale on the Factom Discord server, has long provided thorough and valuable contributions to community governance. WB is joining Factoshi to enable him to better continue that governance work from the position of a standing party.

I am looking forward to working with WB to ensure the Factom Protocol's governance can continue to be improved and refined.
Current Update: Factoidd update: major release for Factoshi's accounting script
Factoshi is pleased to announce a major release for Factoidd, our FCT accounting script.

This release includes the following new features:
  • The spend subcommand: add a spend transaction to your local CSV and bitcoin.tax using the transaction ID.
  • Address aliasing: add an alias to your addresses for easier searching and readability.
  • Config questionnaire: create your config via an interactive questionnaire.
This release also includes major stability improvements (finally!).

Note: it will not work with existing CSV files. If you want to upgrade your CSV history, you will need to start from scratch. Please get in touch with me on Discord if you need help upgrading.
 
Top