TheGraph integration

#1

TheGraph integration

The purpose of this post is to discuss potential ways to integrate TheGraph as an alternative data source for Aragon apps.

Rationale

For now each Aragon app is required to fetch / reduce its state locally through background scripts. These scripts watch for events fired by the underlying contract to extract data out of it, cache these data and pass them to the frontend. These architecture is pretty powerful because it is fully decentralized: the only thing an app need to fetch its state is an Ethereum RPC-endpoint. However, this process is very slow: during the last ANV some people had to wait more than 20 minutes for votes to show up in their browser.

TheGraph is an alternative infrastructure working mostly the same way but in a more centralized and efficient fashion: a node listen for events fired by a contract, extract / format data out of it, store these data into a database and provide a GraphQL access point to query these data. The main advantage here is efficiency: the events are indexed once and queried multiple times. This way, querying votes for governance.aragonproject.eth would take less than 1 second. The main drawback is centralization: the node providing indexation can be hosted by a custom server or hosted by TheGraph but in each case the user has to trust this endpoint. TheGraph is currently working on decentralizing their indexation system through a consensus algorithm. Though, this decentralized indexation system does not exist yet.

Thus, the goal of this post is not to propose a replacement of the current local architecture by a remote TheGraph architecture but to consider a way for app developers to provide end-users two ways to fetch data: one decentralized but inefficient / the other more centralized but more efficient. End users could then choose which solution to use - through a toggle button - depending on whether they prefer to enforce efficiency or decentralization.

Demo

Until recently TheGraph was only able to watch events for solely one contract - which was not really useful because it required to deploy a new graph for each instance of an application … Recently, TheGraph has introduced the ability to handle dynamic data sources i.e. the ability to dynamically add new contracts to watch. One can watch a ‘factory’ contract, identify every new contract instance deployed through this factory contract events, and dynamically watch these newly deployed contract instances to index their events.
For experimentation purposes I have built and deployed a custom graph - relying on this dynamic data source feature - to index aragonPM repos. You can find the source code here and query the deployed graph here. I advise you to test the following query:

{
  repositories(first: 15) {
    appId
    name
    address
    versions {
      version
      contract
      content
    }
  }
}

Architecture

Now there is two ways for Aragon to integrate TheGraph. One being DAO-centered the other being app-centered.

DAO-centered architecture.

In this DAO-centered architecture we would deploy only one subgraph listening for Aragon DAOKit contracts. Through this DAOKit contracts, we could watch every Kernel contract deployed. And through this Kernel contracts, we could watch every known app installed [voting, finance, etc.] Thus, through a recursive dynamic data source system, we could index in the same database:

  • All DAOs deployed
  • All core apps: vote, finance, permissions, etc.

This architecture would make every DAO deployed through Aragon DAOKits queryable for every core apps. The main limitation would be that third-party apps could not be indexed this way [or would require third-party devs to open a PR on the Aragon subgraph each time an update is made which would turn the dev pipeline into a nightmare]. Another issue pointed by Brett is the Big Brother effect … :smiley:

App-centered architecture

Another solution would be to let each app devs deploy its own subgraph for their app. This solution would allow for way more flexibility regarding the dev pipeline. It would also let each app dev be responsible for the implementation of their subgraph [which is a good solution given that they are the ones knowing their app the best]. The trick, though, would be that there would then be no direct way for this subgraph to know when a new app instance [or proxy instance for that app] is deployed to watch it … One solution would be to deploy a network-wide contract with solely one function: function newAppInstance(bytes32 _appId, address _appProxy, address _kernel) { emit NewAppInstance(_appId, _appProxy, _kernel); }

This function could be called by every apps during initialization or even called transparently by the Kernel when one deploy a new AppProxy. Apps subgraph could then watch for this contract to identify newly deployed instance of their app and listen to them to provide indexation.

Discussion and implementation

I think this feature could definitely enhance Aragon end-users experience. I have a strong preference for the App-centered architecture on my side but I think it’s important to let the community discuss it … Depending on which solution is chosen we will work on a more detailed implementation plan and probably try to provide a sample implementation for the upcoming Apiary app.

10 Likes