ESD Funding Request: On-chain Data Audit & North Star Metric Investigation - flour.works

Tagging @mroldann here who runs Aragon’s analytics and is working with various other members from AN DAO on data initiatives.

1 Like

Hi!
It’s great to see such a talented team wanting to help Aragon! I’m open to help/questions at patoR#7041

I like the idea of having documented all our events as we don’t have anything like that yet. It will reduce the overhead whenever trying to get new data from our contracts.

In order to better understand the scope of your proposed work (on top of what @ramon already asked):

  • Will you focus in Aragon Client or also include Aragon Govern?
  • Do you have and example of a data audit?
  • How familiar are you with Aragon codebase? How much help do you think is needed?

Also, I am making public our internal dashboards in a separate post in order to share what we already have in place and what we are still lacking.

@lee0007 - It’s my fault that this vital metric is not available to you so far. I’m still learning how to engage with the DAO and community while delivering operative demand

2 Likes

We are all learning here, that’s why its fun and why data is so important.

1 Like

As promised, sharing NSM dashboard here.
Happy to talk about how to improve it!

1 Like

Hey everyone - thanks for the replies and questions. I will try to answer them here.

Our intention was to focus on all smart contracts.

The best public example is here, however this dash is a bit specific to Set’s needs, and so would Aragon’s. And this would not be the full deliverable - as written, we intend to provide written mapping of all of the event data for Aragon smart contracts.

Not familiar currently, other than reading through the Aragon docs. I think a call or two with a dev and/or product manager that has full understanding of the smart contract system would go a long way.

I don’t think this proposal is intended to re-vamp / solve all current analytics challenges. It is narrow in purpose for a reason. So yes, we would only focus on chains that are indexed on Dune to start - but as part of the data audit, we can investigate options we may have to target data outside of those chains (Metis & Harmony).

Totally understand. I am aware this work is going on, but obviously am not in a place to judge the extent to which working with the current smart contract system is going to extend to your new product. My hope would be that it would be valuable and provide a foundation for analytics on the new product.

Exactly. We intend to “evaluate potential definitions” for the NSM. Essentially, map out what data (on-chain data specifically) is available to use in this definition, and craft a set of potential definitions. We can also provide a recommendation, but ultimately the final decision regarding the NSM would fall on collaboration with those within AN DAO.

This proposal focuses specifically on on-chain data. However, we have full flexibility to work with off-chain data that Aragon is generating. We just have no insight into what that data may be or look like at this point. But we can amend the proposal to also consider off-chain data that your products are generating if that is helpful.


To level set, we intend to be flexible and transparent. We put this proposal forward with the understanding we had at the time regarding the needs AN DAO might have - but we are really just here to offer our analytics skillsets and expertise to help AN DAO wherever it is necessary. So if that means amending this proposal, we are happy to work with y’all on that.

Thanks!

2 Likes

Thanks for the answers @jdcook

What is still unclear to me is what will be that aggregated value upon delivery, on top of what we already have today (and maybe the actual problem is that is not super clear what we already have). To hopefully make this a little bit better, here are some resources that already exists:

I want to reinforce that I am totally in favor of having more people looking into improving how we handle data and making sure more people have access to important KPIs - But would be great to have more clarity on how the deliverables of this proposal will move Aragon towards this.

Maybe an option can be to split the proposal into a first “small” deliverable that brings this clarity, and then if the deliverable makes sense for the network as a whole, work on the larger milestone.

Regards

ESD General Meeting Monday 18 April UTC 1900: Funding approved for 6000 USDC at delivery of

  1. On-chain data audit and report
  2. Smart contract decoding
  3. North Star metric investigation

Due to @daniel-ospina absence at today’s ESD General meeting, this proposal has majority support from ESD. Funding is approved on the basis that further questions and clarification may still be sought by ESD members and the community as we look to understand your work. Congrats @jdcook and Flour.DAO team we look forward to learning from this project

cc @fartunov

@lee0007 @fartunov
Can you help me understand:

  • How this work was approved in lieu of Ramon’s unanswered questions, especially when it provides relevant NSM data from the core team?
  • What is the answer to Ramon’s first question of what is going to be provided above and beyond what we already have?

In reviewing the current thread I am utterly confused what the basis for approving this work was.

I’d also ask that when approving proposals that require the time of core team members that this is explicitly expressed in the proposal. It is completely reasonable in this scenario to expect core team time towards this task but I/they don’t know what that is and the core team suffers the same capacity constraints as members of the ESD

1 Like

All good @b3n you’ve raised several points

I 100% agree that proposals requiring the time of core team members should be “explicitly expressed” as per this proposal

Which returned the following positive and I understood supportive responses from product and analytics leads

Here’s a summary of the proposed deliverables detailed in the proposal, can you indicate which of these we “already have”

  1. On-chain data audit and report
  2. Smart contract decoding [Dune]
  3. North Star metric investigation
  4. Subsequently an artifact for internal/community analytics members to use to build on behalf of Aragon…"

Next, let’s note that the forum will never be the sole basis upon which decisions are made especially when in proactive communication with teams during the development of proposals and where proposal review is requested prior to posting to the forum. I am always happy to review and provide feedback on proposals prior to posting in forum. People only need to ask, this has always been part of my ESD role to date.

We also have the ESD General Meetings and our professional experience as a basis for making decisions. I will share here my conversation with @ramon earlier today in response to his most recent question above and following the ESD majority (2/3 due to absence) funding decision.

This IS the first small deliverable…It is aligning with AA that always proves the challenge hence this being 6k proposal from a world-class team that earns collectively more than that in a day. @mroldann shared the analytics dashboard…And while data studio looks great it is a retrospective metric, nothing there helps to inform future-facing [ESD] funding decisions.

I am a data-driven and business performance professional. Data is the basis upon which I have and continue to manage - millions of dollars in performance marketing budgets over the past decade - and right now ESD and AN DAO simply do not have the data required to make data-driven decisions about the future of [funding] the network.

We [ESD] have different questions to answer than the Product team and we are trying to align around the NorthStar and also operate with some autonomy, sans reliance on what I understand are already very stretched resources for analytics within AA.This is an effort to support analytics and be proactive, to seek answers ourselves, which could provide us all insight.

Given Defi DAO are the largest reference audience for @Harry proposed GtM - [imo it would] be good to align with ecosystem talent beyond AA. Can you please help to empower the DAO in our need for data. Your support is mission-critical, your forum post will help to inform [proposed] work [&] I very specifically am not looking to rehash exisiting data points but to seek other [SMART] relevant metrics [as they relate to the the NSM]

I’ll try to frame my need for data another way

Given “active” number of DAO as currently measured using

If these were the ESD success metrics then I would simply fund everything to generate proposals and increase “active” metrics for the network. But in my professional opinion that would amount to negligence, a breach of the duty of care entrusted to me as a member of the ESD for the strategic funding of network growth.

And so I have sought the help of some of the most talented analytics minds I know to help surface, investigate and model data related to the NSM in search of a more refined measurement of funding success. The current process is simply too subjective. I want and we [the community] need data to help inform funding decisions

1 Like

Thanks, it’s clear now there is more context (and a direct but unseen response to Ramon’s questions) which is why i asked.

Regarding this part: "

Explicitly expressed to me would be X hours/days

(A bit late to the conversation but) very happy to see this proposal pass!

This is much needed, and the ‘active DAOs’ definition is pretty tricky. Even if it’s something we’ve been thinking about and wrestling with, a) having another (expert) perspective on this will no doubt improve the thinking, b) having a deliverable for it, within a short time frame, is way better than not having a metric!

Kudos to @lee0007 and the Flour.works team for proposing a pragmatic and bitesize first step, I hope it can mark the beginning of a longer collaboration.

(P.s. I’d love to join the kick-off of this initiative if possible - I am working on product marketing at Aragon and NSM helps defining which part of the market to pursue).

3 Likes

Glad we have the proposal through!

Stoked to have some web3 native insights (Dune dashboard) where everyone can validate the network’s growth and health!

A few notes:

  • Based on the original proposal, there is little to no work required from the AA/AL product teams to facilitate the delivery of the proposal. I have expressed my support based on that assumption?
  • As per the comments shared, ideally, we would want to see data across all Dune-supported EVM chains where Aragon is live. Is that the case? [clarifying question, won’t change me supporting the proposal]
  • With respect to “active DAOs,” - in addition to the ambiguity of what “active” means, overall the metric has many disadvantages. Ideally, we would want to see some exploration on whether a better metric can be derived based on available data and your experience and expertise with other projects [i.e. if everyone is measuring A and we chose B, we should have a great reason for doing so, or we should revert to also using A]

@jdcook, does the above make sense?

2 Likes

@jdcook can you please respond. I understand you had a call with @ramon to redefine the scope. Could you please share some transparency

After chatting with the Flour.works team, here is a list of points that would be great to have their help on top of what @mroldann already created.

  • Get all app (addresses and names) used by Aragon Client DAOs, and the most used methods/functions for each of them
  • Get all DAO names from Aragon Client
  • Some finance transactions are not triggering events and need to find out why. @mroldann will follow up on this below
  • Documentation for all contracts
  • Dune decoding

Resources to help with all the work

1 Like

Hi there, completing what @ramon said.

Currently, our Vault subgraph does not show complete information for some DAOs/tokens for both Balances and transactions .

Example from Decentraland DAO

  • Left: balance according to subgraph
  • Right: real balance from Etherscan which coindices with Dune

We would like help to find a way in Dune to fetch all DAO’s USD AUM evolution and transactions with queries that don’t timeout.

1 Like

Hey @fartunov - sorry for the delay. We had a good chat with Ramon and Martin - thanks @ramon for providing links to all the resources.

We have been investigating the smart contract system and are working on documenting and mapping out events as per our first deliverable (referencing deliverables below).

@fartunov to answer your questions here:

It is our intent to focus on mainnet for now - the reason being is there is no new discovery we can make by extending to other chains. We would be happy to roll a future proposal to help with decoding and actual metric building on other chains once we have this foundational audit and investigation completed.

We can definitely include that exploration in our North Star Metric investigation. Happy to ideate on some other potential NSM’s to consider other than “active DAOs” based on the available data and other metrics the industry is favoring.

As per the list of things that @ramon and @mroldann have suggested (below) it would be great to have our help with - glad to see this!

As I mentioned in our call with them, our goal is to be a resource - an extension of the analytics and product efforts already happening. So it is great to see the uncovering of some analytics needs right out of the gate. However, I want to be clear we will only be focusing on the 3 deliverables as a part of this proposal. We are hoping that these deliverables uncover other projects / tasks that we can help @ramon and @mroldann (or other teams at Aragon) tackle. We would plan on submitting another proposal for funding for these items outside of the scope of the current deliverables.

3 Likes

Thank you! The clarification is much appreciated!

Hello all - I am reaching out to notify that flourworks will not be moving forward with delivering upon this proposal. This decision from our team is coming from a combination of:

  • lower bandwidth than initially planned
  • higher complexity than initially planned
  • lower value to Aragon than initially assumed

From a bandwidth perspective, we have not been able to dedicate the time needed to deal with the complexity of the Aragon smart contract system. That is totally on us - we underestimated the difficulty of getting up to speed on the inner workings of Aragon’s smart contracts. We feel it would need quite a bit more dedicated support from internal team members for us to get to a place where we could properly outline and investigate all the on-chain data that Aragon contracts are generating.

Because of this general state of complexity and us being behind, it is apparent to us that this project is not likely to drive a lot of value to Aragon as originally proposed. We feel that a North Star Metric investigation would be best done by team members already ramped up to the smart contract system - there are just a lot of nuances to the data that are difficult to handle without built-up context.

Thank you to everyone that worked with us on this proposal and initial exploration in any way (especially @ramon @mroldann @lee0007 ).

4 Likes

@jdcook thanks for the transparency and honesty here. Hope that we can have a better structure to work together in the near future.

1 Like

ESD General Meeting Monday 18 April UTC 1900: We would like to thank this team @jdcook et.al for your professional integrity. It would have been both difficult and time consuming for us to understand if the final deliverable was to the standard proposed.

As the project did not proceed the approved 6k funding will not be released. However, in recognition of the time taken to deliver your professional assessment, the ESD unanimously agree to reward 5% ($300 USDC) of approved 6k funding as per the Scout Program. Thank you for your help, we would welcome future opportunities to work with this team.

@fartunov @daniel-ospina
cc Finance @Ricktik6

1 Like