Grant Update [BIF-Factomatic-001] Factom DIDs

Secured
#2
Update 1:

This grant application promised to bring DIDs to the Factom blockchain. The deliverables included:
  • a DID method specification
  • an open-source resolver for DIDs
  • an open-source registrar for DIDs
  • a self-hosted API for creating DIDs, with the first 1 million DIDs created via the API, covered by Factomatic
The specification for the Factom DID method is available here: https://docs.google.com/document/d/1F6IPOsANXmvGY1xmzXiWVRD8O5AVCFTBFBHcmU2WCZo/edit

== BIF ==
The Factom Java client has a new module that allows you to directly work with DIDs from Java, without having to use the universal resolver and registrar. This will be release soon together with documentation updates and support for IoT signing created by TFA.

The universal resolver is almost finished, and we are working on the documentation and config options right now. After that we will ask it to be merged into the official universal resolver project.

We are also working on getting the current proposal for DIDs into the Factom Improvement Protocol (FIP), as we were made aware of another DID implementation in the works that currently is incompatible with the DID work done by the Factomatic and BIF for the public mainnet. The other implementation decided to go ahead and implement their own DID implementation called “did:factom” to add to the confusion. Currently BIF is looking at the impact and viability of merging the two implementations in the future into a new version. We want to do this using FIPs, to make sure everybody will be aware of the changes and process and have uniform solutions if we want to become the data integrity and identity standard. In our belief this clearly shows the need for FIPs.

We expect to release the resolver and Java client integration for testing and release for the general public before the end of May 2019.

== Factomatic ==
The deployment of the registrar is currently live on testnet and can be seen here: https://factomatic.github.io/factom-did-ui/.

The registrar allows the creation of:
  • Ed25519 key pairs
  • ECDSA (over secp256k1) key pairs
  • Services associated with the DID
  • Authentication keys associated with the DID
  • Encryption of the private keys for the DID
The registrar has two modes of operation: a basic mode, which allows a no-fuss creation of a simple DID with a single key pair and a more advanced version, which allows choosing between different key types, distinction between authentication and non-authentication key pairs and registering of services, associated with the DID.

The code for the registrar is open-source under the permissive MIT license and available here: https://github.com/factomatic/factom-did-ui

Behind the scenes, the registrar uses a rate-limited HTTPS endpoint to provide security and to protect against flooding attacks on the Factom blockchain. We are currently working on extending the registrar UI to allow modification of existing DIDs, an extra functionality outside of the scope of the initial proposal. We believe we will have this completed within the next two weeks.

Overall, we expect to move to a mainnet version of the registrar within the next 2-3 weeks and we will be funding the first one million DIDs created via the API, as pledged.

In addition to this, we have implemented a web signer Chrome extension -- also outside of the scope of the initial proposal -- which allows importing of the DID keys and using those keys to sign arbitrary content, as requested by some website. The code for the plugin is open-source under the permissive MIT license and is available here: https://github.com/factomatic/web-signer-extension. We will be publishing the extension to the Chrome Web Store within the next 2-3 weeks.
 
Secured
#5
@Niels Klomp , Valentin, during grant discussion you told about an API to work with DID. In my opinion there should be an open-source API, which everyone may install and use on own server.
If I want to implement DID Into my application, someone's proprietary API is a huge red flag.
Are you going to provide open-source API?
Hi, Anton! Indeed, we promised to release an open-source API. This is the same API that is currently being self-hosted by Factomatic and is used in the registrar UI. It will be open-sourced before the work on this grant is officially completed.

Even with the current closed-source state of the API, however, I'm not sure why you feel that using the API endpoint we are providing should be a huge red flag. Could you please elaborate on that?
 
Secured
#10
@Anton Ilzheev -- I think that the confusion here stems from the fact that you referred to a "red flag" because the API is self-hosted and not open-sourced yet.

However, I don't agree that it is a red flag and IMO it's perfectly safe to use it even while it is closed-source and self-hosted. I will try to clarify this below.

I believe the reason why you consider it a problem might be because it's not entirely clear what the API does. Essentially, we have a REST endpoint, called "/write-did" implemented in Python using Flask which:
  • accepts JSON payloads
  • uses HTTPs
  • has rate limiting (via nginx)
  • and will also have JSON schema validation for the payloads (we're finalizing this right now)
This endpoint doesn't do anything, which is critical. It takes the DID document, as the JSON payload, and it records it on-chain. That's it, really. The rate-limiting and JSON schema validation are essentially protections because it's an open API endpoint that anyone can access, so we want to avoid:
  • flooding attacks on the blockchain
  • someone using Factomatic's ECs in order to records entries, which are not DID entries

The DID document that is sent as a payload to the API can be seen in the last step of the DID UI registrar (available here: https://factomatic.github.io/factom-did-ui/). It doesn't contain any sensitive information and the entity hosting the API doesn't gain anything from modifying the payload. Everything, which is sensitive -- in particular the public/private key pair generation & encryption -- is done client-side in the UI.

The only attack that Factomatic could potentially carry out is a censorship one. However, I also want to point out that we pledged in our grant proposal that we will sponsor the first 1 million DIDs created using our API endpoint and I don't think it's possible to do this without having a self-hosted version of the API.

In the end, we will release the API as open-source, but it's nothing fancy, so I am not sure you (or anyone else for that matter) would gain much by self-hosting it, instead of using the version we will provide (which will also be free, so it will spare you some server costs + EC credits).

The "interesting" bits in the API are the JSON schema validation and the rate limiting, however, you wouldn't really need this if you're hosting the API yourself and it's accessible only to you... So, in the end if you end up hosting a version of the API, it will essentially serve as a relay of JSON payloads from the API endpoint to factomd.

Hope this clarifies things a bit.
 
Secured
#11
That update is looking good, thank you. I will start looking again at the details of DID based on your docs.

I still have to comment on the fact there hasn't been any update for five months, or more critically no way for standing parties to even ask for an update as this thread didn't exist. The grant system is currently rather messy and very hard to track, so having two of the most preeminent ANOs being exemplary in that regard would have gone a long way helping all of us. We can actually already see some regrettable consequences, such as HashNStore grant being called our for using Factom Inc. identity standard instead of DID. I'm sure they would have been very happy to go with DID and collaborate if you had communicated your progress over time, given us some visibility of when and what we should expect to come out. So for the sake of transparency and efficiency of the ecosystem, I'd appreciate if you could be more transparent on your community funded efforts in the future, many people care (even if they don't voice it), and I certainly do.

Again, thank you for your work.
 
Secured
#14
Yes. Although it might seem easy to say the following at grant proposal time. The lack of our updates on Grants is something we discussed in the Sphereon management team recently and is something we promise to really improve. I do hope people see with for instance our quarterly update we are doing everything in our power to make a success of the public protocol both on the technical, solutions and clients level.
 
Secured
#15
I agree with you, Paul and Julian. Indeed updates should have been handled much better and overall communication with most of the community was lacking. I did reach out to several members of the community for early feedback on the registrar UI, but I don't want to use this as an excuse of any sort. Grant updates should have been provided.

I want to assure everyone that for any future grants that we will be awarded we will have the appropriate level of transparency during development.
 
Secured
#17
Hi Valentin,

Thank you for this update and for you work. It would of course have been nice to hear about the progress of this grant before. Thank you for committing to improve this aspect in the future. This can avoid waste of time and money from many different parties.

Here are a couple of questions:
- What are the main differences of your DID proposal with the W3C reference one (https://w3c-ccg.github.io/did-spec/)? For example, is the synthax used in your proposal for updating a DID in line with the exisiting W3C standard?
- May the W3C spec dramatically change in the future ? How confident can we be that your proposal will still be compliant with the W3C standard in the future?
- Based on
We are currently working on extending the registrar UI to allow modification of existing DIDs, an extra functionality outside of the scope of the initial proposal. We believe we will have this completed within the next two weeks.
, do you intend to provide other functionalities within your registrar such as Identity update for example?
- Is there any hierarchy options/possibilities in your DID proposal?
- In what language is the resolver written?
- What will be the main methods/functionalities proposed in this resolver? (e.g. "GetValidPublickeys")
- If the OpenAPI FIP grant proposal passes, will you try to standardize the methods used in the resolver with this FIP?
- Could you let us know why it has been impossible to find a consensus with Factom Inc about DIDs and why you currently consider the 2 standards incompatible? When you look at them, they look quite similar.
- If a new V2 DID standard is created, will you update both the registrar and the resolver?

Thank you for your time.
 
Secured
#19
As was mentioned we expect to release it before end of the month and to be clear that wasn't part of this grant. Yes it will contain documentation.

It will be part of a bigger release that contains other fixes and improvements for the client. Most notable IoT-SAS support and we hope to have the FAT Java client included in that release as well.
 
Secured
#20
Hi Matthias,

I will try to answer your questions inline below:

- What are the main differences of your DID proposal with the W3C reference one (https://w3c-ccg.github.io/did-spec/)? For example, is the synthax used in your proposal for updating a DID in line with the exisiting W3C standard?
The DID spec that you linked to doesn't define any specific syntax for creating, updating or deactivating a DID. Instead it outlines what kind of functionality should be provided by an implementation and what kind of information should be present in a DID document. What the spec provides in addition to this (through the use of the @context field) is a summary of the JSON fields, which have a special and known meaning in the context of a DID (e.g. see here: https://w3c-ccg.github.io/did-spec/contexts/did-v1.jsonld).

Given this generic specification for a DID, one can write specifications for a so-called DID method. DID methods are the mechanism by which a DID and its associated DID Document are created, read, updated, and deactivated on a specific distributed ledger or network. A DID method is the incarnation of the DID standard on a given blockchain.

The goal of this grant is to provide a DID method specification for the Factom blockchain, as well as the corresponding implementations of a resolver and registrar. In our DID method specification, resolver and registrar we have support for all major functionality of a DID: creating, reading, updating and deactivating. In our JSON representation of a DID document, we have also used the same fields, which are defined in the default DID @context. There aren't deviations from this.

The only place where we have a slight deviation is in the optional proof field, which is talked about in the W3C DID spec. When updating or deactivating a DID, we don't put the proof in a proof field, but instead put the signature in an ExtID of the update or deactivate entry. Please see the spec I linked in the original update post for more details and let me know if you have any questions. The reason we chose to do it this way, is that putting the signature -- covering the content of the actual entry -- inside an ExtID (instead of as a separate field in the entry itself) has become a common "design pattern" for a lot of projects: Factom Identities, the initial Standing specification from Paul Snow, on-chain voting, FAT, as well as the current Standing specification I'm working on for the on-chain governance working group.

- May the W3C spec dramatically change in the future ? How confident can we be that your proposal will still be compliant with the W3C standard in the future?
Everything is possible, including a dramatic change in the future, although I deem that highly unlikely at this point. While the specification is a Community Draft, it has already undergone a number of revisions and has been around for the good part of 1.5 years. In addition to this, it has already seen a significant buy-in from a non-negligible number of projects, including Bitcoin and Ethereum (see here), so there is a clear disincentive for the editors to make drastic changes, as this means a lot of projects will have to update their specs & implementations. We are definitely committed to updating the spec & implementation if there is a shift towards an updated DID standard, but I don't foresee that happening.

do you intend to provide other functionalities within your registrar such as Identity update for example?
Could you clarify what you mean by an Identity update?

- Is there any hierarchy options/possibilities in your DID proposal?
Could you clarify what you mean by this?

In what language is the resolver written?
BIF is working on the implementation of the resolver and it is being written in Java. At Factomatic, we are working on an additional resolver implemented in Python, which is outside of the scope of this grant.

- What will be the main methods/functionalities proposed in this resolver? (e.g. "GetValidPublickeys")
I would let @Niels Klomp provide details on that, but in general the resolver takes a DID as input and returns the up-to-date DID document for the DID. From the up-to-date DID document (which is plain JSON), it's trivial to extract any keys or other metadata pertaining to the DID.

- If the OpenAPI FIP grant proposal passes, will you try to standardize the methods used in the resolver with this FIP?
I haven't looked into this proposal in detail yet (or any other for that matter), as I'm currently still reviewing ANO applications before delving into the grant ones... However, from what I can tell this proposal is dealing with the Application Identity standard, which is not really compliant with the DID one. It is an unfortunate situation, for which I personally shoulder a lot of the blame due to the lack of updates on this grant. However, I strongly believe that we need a unified standard for digital identities and I strongly believe that this standard should be based on DIDs. As such, the development efforts on digital identities from Factomatic are and will continue to be centered only around DIDs in the foreseeable future, with potential extensions leveraging ZKPs.

That said, I cannot speak on behalf of BIF on that matter and they are the developers of the resolver under this grant, so I would defer to @Niels Klomp for his view on that matter.

- Could you let us know why it has been impossible to find a consensus with Factom Inc about DIDs and why you currently consider the 2 standards incompatible? When you look at them, they look quite similar.
We haven't had any concrete discussions with Factom Inc about a common standard. Factom Inc has expressed their interest in finding a common ground between the two approaches, as they currently deem them to be incompatible. However, I personally have not seen the factom:did specification, so I cannot comment on any differences between the two. IMO, the generic DID specification doesn't leave a lot of room for major incompatibilities between the two DID method specifications, so it shouldn't be too difficult to come up with a common standard.

For the sake of transparency, I want to say that Factom Inc have clearly stated that they do not want to provide a competing solution and that they want to converge on a single approach. However, at this point, they have opted to implement their own standard due to business reasons. In my personal opinion, this is an extremely unfortunate situation, which has caused a lot of unnecessary confusion and is a result of the lack of timely updates for this grant and the (IMO) late outreach regarding this topic from Factom Inc.

- If a new V2 DID standard is created, will you update both the registrar and the resolver?
As I mentioned above, we are strongly committed to DID as the standard for digital identities on Factom, so if there is DID method specification recognized by the majority of the Standing Parties we are happy to converge to that.
 
Last edited:
Secured
#21
Thank you very much Valentin. You and your team are one of the best asset we have in our ecosystem.

Could you clarify what you mean by an Identity update?

Could you clarify what you mean by this?
I should have said : Key Identity Update or Key replacement.

For the hierarchy I wanted to refer to the hierarchy of the keys and whether a key of higher level can replace another one from a lower level.
 
Secured
#22
Thank you for the kind words on behalf of Factomatic, Matthias! Much appreciated.

Regarding your questions:

1. You can definitely do key replacement with the spec that we have developed and you will be able to also do this (for paper DIDs) via the registrar UI: https://factomatic.github.io/factom-did-ui/. This is the "Update Existing DID" action, which is currently under development and should actually be done by the end of this week. In fact, with the approach we have outlined, you only need a single on-chain entry to add or remove multiple public keys, authentication keys or services.

2. Regarding key hierarchies. To the best of my knowledge, this is not something that exists in the generic DID specification (https://w3c-ccg.github.io/did-spec/). However I think you should be able to accomplish the same with delegation, or we could also extend our DID specification to allow for ordering. This would require some changes to the registrar and resolver, but it's nothing major IMO. If there is substantial demand for it, or you need it for some specific use case, we could look into that.
 
Secured
#23
2. Regarding key hierarchies. To the best of my knowledge, this is not something that exists in the generic DID specification (https://w3c-ccg.github.io/did-spec/). However I think you should be able to accomplish the same with delegation, or we could also extend our DID specification to allow for ordering. This would require some changes to the registrar and resolver, but it's nothing major IMO. If there is substantial demand for it, or you need it for some specific use case, we could look into that
I think it could be an interesting feature to add. Happy to discuss this further.
 
Secured
#25
Hey @Anton Ilzheev! We have had several meetings with @Carl and @Sam Barnes in the last couple of weeks discussing a v2 of the DID specification, which should combine the work Factom Inc did on the "did:factom" method and the work we have doing on the "did:fctr" one.

We figured it doesn't really make sense to release something when 3-4 weeks later we will change the spec and the implementation, so the work on the DID grant will be "re-based" onto the updated spec. I expect we will have a preliminary version of the specification available by the end of this week or early next week at the latest. We will give this to the wider community for review and feedback and once it's finalized we will have to accommodate the changes to the resolver and the registrar.

Let me know if you have any other questions. In the meantime, you could also follow some of the ongoing discussion in the #dids channel of the FCD Discord.