[SPHEREON-003] Core Development Continuation grant proposal

Secured
#2
As promised we would make sure the 2 FTEs are filled right from the start.

We currently have the following 3 people working on Factom core. @Sander Postma, Gabriel Kidane, Laurens van den Brink.

Sander is currently looking into coinbase related security caps. He has been working on fixing the brain swap crash and some other findings in the code. For instance he fixed the closing of the database whenever a crash occurs and added better runtime state control for the factomd node.

Laurens has been researching and running local networks and in talks with Sander about next steps. We expect some first results in actual code from him real soon.

Gabriel is starting to get up to speed on Factom.
 
Secured
#5
Please note that this report is highly technical because of the nature of the work done by our devs, which has mainly been hunting for Bugs and fixing them in factomd, based of several Jira issues provided by factom inc. concerning brain swap behavior and failing unit tests. Most of the time last few weeks is spent on the LiveAPI (see last part)

FD-922: Fixed 2nd pass flagging (EntryDBHeightComplete)
  • Found & fixed some problems in entrySynching.go

FD-845 – Brainswap audit panic
  • improved handling of heartbeat panic
  • fixed and refactored AuditBrainSwap unit test
  • improved shutdown/db close handling in the validatorloop.go
  • introduced sequential run-states to have fine grained control

FD-1037: additional brainswap tests
  • Observe behavior when two audit nodes with the same id are put on the network deliberately.
  • Experimented with some extra failsafe routines

FD-883_failing_back_to_back_brainswap
  • Fixed minor issue in the test code
  • Found & fixed a problem in processList.go: Don't panic when processing an ack message from the future

FD-809 - Inflation limit (still open)
  • Researched & tested possible solutions for inflation limit capping
FD-887 Simtest failure
  • Researched why certain tests fail on some systems
  • Made it reproducible by running stress tests along with the go tests
  • Determined that the cause is IO throughput when simultaneously writing databases and debug logs for multiple nodes.
  • Devised a fix using shared memory files (/dev/shm) as temp storage for those files and successfully tested this using Circle-CI.


LiveAPI

Working on a new “Live Feed API” which enables functionality in factomd to send out events. The support by factomd is very low level with very low performance impact. A second layer is implemented to open op the feed’s data through a more comfortable API platform where subscriptions and filters can be made on the feed.
  • LiveAPI features kickoff & design
  • Wrote prototype for benchmarking the first layer (factomd side)
  • Wrote testbench setup for a second layer
  • Working on production code & unit tests for both 1st & second 2nd layer
Details on https://factomize.com/forums/threads/factom-live-api-considerations.2142/
 
Secured
#9
This grant is being continued as SPHEREON-007 CORE DEV and we'll put our updates there.
I understand that further technical updates will be posted in Sphereon-007 core dev, but I would appreciate an answer to this question I posted above:

Can you make an assessment about this grant in terms of succeedding relative to the succsess criteria you posted in the application? I.e. did you provide the developers and manhours you applied for, and now deem this grant complete?
Thank you.