Real-time Community Health Analytics

I like this proposal, keeping the pulse on community health is definitely important for a DAO. @noturhandle can you please clarify in what form the real-time data delivery will be? Like, will it be a spreadsheet with data populated by some monitoring service, discord bot or web page or smth completely different?

2 Likes

Having Community Health metrics is a need every DAO has.
This research and tool will be pretty useful for all Aragon DAOs and web3 space as a whole.
Happy to chat/help when needed!

2 Likes

Thank you @noturhandle - a great initiative!

A few clarifying questions:

  • Could you please clarify what is the actual deliverable - are you setting up the tools in place so that the network can track those health metrics in real-time (automated / semi-automated data collection and insight synthesis) as the title and budget breakdown suggests, or is it a one-time snapshot assessment (as the current deliverables description suggests)?
  • Huge supporter of the collaboration initiative in general.Kind of unclear if it’s a collaboration across multiple DAOs why is Aragon Network carrying 30% of the cost? I would recommend working off the best practice of the DAOstar one financial proposal, namely AN DAO contributes 10% of the total funding and its conditional on at least 40% of the total budget is secured (the 40% includes the 10% by Aragon).
  • Is this framework going to be a public good to the web3 space to be used by anyone at no-cost or is the funding going towards the development of a product/service that rnDAO will monetize, if the latter this feels like it should be more viewed as a first-check investment in rnDAO, rather than a grant as currently portrayed

Looking forward to your responses as I have follow-up questions but wanted to make sure I understand the proposal well before going into more depth

4 Likes

Welcome @noturhandle! Many thanks for the proposal I am sorry my feedback is coming late as @daniel-ospina did share it with me in advance for transparency. Firstly the initiative is needed for the space.

I second some of @fartunov points would be great to hear if these results will be open source and available to other DAO communities.

Will there be any levels on the budget set for different groups, you define custom (I can handle this) but would be good to have transparency on the levels being charged to other communitys. Also can we set a point at which you would like to see funds released eg Work requires 90,000 and only one other DAO participates, can the same deliverable be met with $60,000 dollars? One line such as the bellow on the conditions that would be needed for funds to be released would be good.

I also just wanted to point out that a proposal for season 1 data insights team in the ESD was rejected at similar funding levels which had some similar goals, just want to make sure we are clear and transparent on reasons that was rejected so holders can make an informed decision sharing it here so members can go back and review:Funding Proposal for Data and Insights Squad Season 1

I am happy to see this going to the main DAO.

1 Like

Hi @AlexClay and @fartunov, thank you for your interest in this proposal and your comments.

Replies below (they cross over a bit between points so if doubts remain, please read all the replies. Apologies for the length, but hopefully it’s thorough enough to avoid unnecessary back and forth)

The proposal is primarily an explorative and applied research initiative, with the key deliverable we’re aiming towards being the framework to comprehend community health. To carry out this research, we’ll need to develop a data collection tool (which operates in real-time).

As there are currently no good frameworks on community health for DAOs, we can’t say whether the outputs of the research will lead to a capability to analyse the data in real-time (i.e. automated indicators) while still providing meaningful insights or whether meaningful insights will require ongoing work by a data scientist trained on the framework to be developed. As such, the proposal does not include an automated dashboard, but does include the insights (as we have budgeted for work by Katerina (see team section) and potentially others to explore the data). If we can create a dashboard, that’d be great, we just can’t promise it given how little exploration there’s been to date on the topic. So please understand the real-time component as the stretch goal of the current proposal.

The DAO Star one proposal is one for the creation of standards, and by definition, a standard is only valuable if it is widely adopted. That means that a rather significant amount of work for said projects is dedicated to partnerships and so raising from multiple DAOs, although extremely time-consuming, directly contributed to improving the core value proposition of the initiative. It’s also important to note that a standard is fully a public good.

In contrast, the work we seek to undertake is more akin to deep research, which means that any time spent applying for grants increases the cost of the initiative without significantly increasing its value, and so a larger number of partners reduces viability. Additionally, the benefits are partially useful for DAOs at large (The framework of analysis is a public good and we hope supports the ecosystem) but especially useful for Aragon (or other sponsoring DAOs) as the data analysed is their own and hence the insights tailored to their community.

To clarify whether this is a business or a research grant, we’re seeing this as non-for-profit research within RnDAO. We’re not developing a business plan nor a business model for the project. And the research outputs (and the tool that facilitates the research) will be published open-source.

We suggest a similar model for knowledge-creation leading to innovation as exists currently with researchers receiving grants and endowments to advance knowledge generation (which should be open source as much as possible) and then the next set of actors and institutions (currently VCs and incubators/accelerators) who leverage the broad knowledge base to explore specific business opportunities.

There could be, in the future, a product or service business opportunity that could leverage the knowledge we seek to generate through this research proposal (the same way that a number of ventures over the decades have been built after years of scientific research on a topic), but can’t say whether RnDAO or its members will ever attempt to build this hypothetical venture.

Turning this research into a venture would likely require investment to finance further work and development, and if we were to pursue that (again, we haven’t discussed it to date) I imagine we would reach out to those we have built relationships with to explore that possibility. So perhaps in the future, we can approach Aragon for an investment.

For clarity, we do consider the scenario where this research leads to one or more venture-like initiatives to be ideal. We just can’t say with current insights whether that makes sense or whether if it did, we would pursue it or a different actor would be better positioned to do so. As such, we have designed this proposal to already provide significant value for its sponsors in providing targeted insights and for the broader ecosystem in helping to elucidate a complex topic with huge potential for DAO advancement.

The team for this proposal has accepted to take some risks in its development, and we plan to begin the literature review and build the data collection tool at risk once the first sponsor has been secured. This risk has been considered more palatable than delaying the commencement of the project and risk losing contributors to other opportunities and hence the initiative never happening.

Failing to secure at least a second sponsor would mean that the research team experiences a loss but the basic deliverables are still achieved. We hope we can avoid that scenario and instead make this initiative a stepping stone to encourage further collaborative research in Web3.
Meanwhile, the risk of failing to secure a third sponsor is that of a more limited scope for research, which reduces our chances of achieving the stretch goal of real-time community health analytics but does not put in danger the creation of the framework nor the data collection tool.

We hope the Aragon Network is keen to share some of the risk in this proposal by releasing funding early, and in doing so together we can take a step to foster research and avoid the web2 mindset of dumping negative externalities on researchers.

The Data and Insights Squad proposal was oriented towards figuring out how to measure Aragon Active DAOs and define team KPIs. In contrast, the proposal we’re putting forth here is a research initiative focused on exploring how to evaluate and comprehend the health of a community.

As I understand it, the Data and Insights Squad proposal would facilitate performance tracking for Aragon but would not seek to explore the factors that can predict or transform the ability of a DAO to thrive, equally their proposal would have no direct impact beyond the AN DAO / AA. In contrast, we plan to use the AN DAO as a case study to collect and analyse a wide range of data points related to community interactions and seek to derive meaningful action points that are specific for Aragon but could also have applicability for DAOs at large. Within the scope of the current proposal, we won’t be able to validate the applicability of the insights beyond the grant-giving DAOs but the theoretical framework to be created does have broad applicability.

As such, it is our view that both proposals could have coexisted and event supported one another (note that the reasons of ESD members Ivan and Lee to deny the Data and Insights Squad proposal are not being put in question here).

Because of the mixed impact (for Aragon primarily, and the ecosystem at large secondarily), and Daniel’s involvement within the ESD, we feel this proposal is best presented first to Main DAO as opposed to the ESD.

Please let us know if any doubts or further comments remain :slight_smile:

1 Like

@daniel-ospina @noturhandle could there be opportunitiy for AN DAO data insights contributors to build/collaborate on this project through DAOOps bounty funding? cc @anukriti10

I ask because as @AClay referenced their s1 proposal includes the development of a community dashboard. Although it proposed a different focus (team KPI’s) - early stage AN DAO contributor participation 1) builds capability to advance outcomes of this proposal and 2) could support delivery of “…the stretch goal of real-time community health analytics…”

I think research always has its place within organizations, even more so if it aims to give insight into the situation of social layers. From my point of view, the results should be oriented slightly differently. I like the idea that the research goes to analyze the causes that might lead a contributor to participate in or possibly leave the Aragon DAO. I think this topic is related to people’s needs; these same needs can be diametrically different depending on the category of stakeholders to which one belongs. Research has a potential to meet the different actors within an organization by bringing out data that no single working group would be able to do individually (this partly answers the problem expressed by @fartunov in the sample bias post [On delivery]). I think the real added value of this research would be to ghater and analyze qualitative data allowing each stakeholders category to work with a better overall view of the system needs. There is a chronic need for mutual understanding, heaven more in decentrlaized organisation wich are facing huge challenges into creating and coordinate a shared vision wich everybody can relate to. I think this research would have the potential to bring the various point of view togheter, however the scope of this research sould be oriented slightly into this direction.

1 Like

Yeah of course. Collaboration is most welcomed

1 Like

Thank you @AcidCavin for sharing your views.

In our working definition, Community health includes onboarding and onboarding (or inflows and outflows). So we do plan to look into this to some degree. The idea is first to create the overview (overall framework) and then go into the specific verticals or subtopics.
The pulse survey and interactions data functionality paves the way for a lot of further research, so hopefully this is but the begining of what can be a whole area of study into DAOs and communities!

1 Like

Many thanks for the response @daniel-ospina there were a lot of questions there.

Personally I am happy as a group to participate in the risk. Do you also know who else you are requesting funds from at the moment?

Also happy to see the results are public, appreciate your insight on the venture side. As long as the results are public there should be not issue with this.

Thanks,

Alex

1 Like

Thank you for a very detailed response. However unfortunately it gives “it depends” answers to straightforward yes/no questions, so I will try to rephrase:

  • Is Aragon receiving a tool that can be meaningfully and easily used periodically, or a one-time assessment?

So the research will be focused on Aragon and up to two other DAOs (subject to approved funding)?

We rejected @Brian’s ESD funding request although different in topic, was very similar in terms of work required and somewhat more extensive and complete in deliverables on the basis of somewhat rich valuation for vague deliverables. Based on the same yardstick this one probably shouldn’t pass.

Based on your responses to @AClay and @AcidCavin it seems that @Fabs’s ESD funding request is again an example of “better bang for the buck” from the Network perspective with a similar focus. To clarify, based on your response, out of all stakeholder groups we are aware of, the focus here is on compensated DAO contributors?

The above being said the Network should also support the progression of the space as a whole, for example through the DAOstar proposal [funded] and the GitCoin matching pool [hopefully coming].
However, the response of whether we are funding public good or product development for rnDAO (not going to copy-paste as it’s unnecessarily wordy) falls flat on clarifying which one it is.

If the outcome is a useable public good functional tool available and free for anyone to use, build upon (rnDAO included obviously), and customize I think this would be an amazing proposal. All the “future research” hedging sounds like there are going to be one or more follow-up proposals “working towards that goal”.

The proposal is priced as an investment, presented as public good funding and the deliverables are a one-time assessment - there are some assumptions here that I tried to clarify but your response did not provide any additional clarity.

I have spent time in professional services and understand the business model - it’s a great one. Not sure a network with about 100 contributors (@Ricktik6 can provide the exact figure) should be paying $300 per contributor for assessing onboarding and offboarding experience, considering @Shawncubbedge is already doing strong work on the onboarding front.

For me, it’s a no-go in its current form.
Given the Network DAO design, it makes no sense for me to invest further time in this discussion. The team has enough voting power in its circle of influence to push this through. My role is to do my best effort in spending well the ESD budget. This is the Main DAO over which my influence is a rounding error even at current network participation levels.

3 Likes

Hi,

Congrats and thanks to the team for putting forward such a solid methodology and proposal. Regardless of the outcome here, wishing success and would like to follow the project.

This is not exactly ‘my area’ that I’m contributing to at Aragon (I work on product marketing), but I think it’s important to assess the proposals holistically in terms of where we are at and what our priorities are.

Not sure a network with about 100 contributors (@Ricktik6 can provide the exact figure) should be paying $300 per contributor for assessing onboarding and offboarding experience, considering @Shawncubbedge is already doing strong work on the onboarding front.

100% agree with this. It feels like over-analysis for our state, especially given that we already have community insights coming in (from the sources mentioned in previous posts).

Furthermore, my interpretation is that our community is not huge & will have ‘health conditions’, because we are at a stage of rebuilding the project and preparing to rally as a community around a new product. I imagine that a healthy community depends on people being able to contribute to advance the project, which for now is restricted a lot by the moment of the product. So therefore I have doubts as to how much the insights would turn into action (on our part), if in this phase they are restricted in action, and in the next phase, we’re looking at a different picture. (Like going to the best doctors in the world for assessment on cutting-edge machines for a health assessment, when you have a broken leg).

Considering the community size, the transitional moment we are in, and that there are already related insights on community health (mentioned in posts above), I wouldn’t support the proposal.

3 Likes

Thank you Harry for your feedback.
I’m sensing that we haven’t done a good enough job in explaining the power of ONA and the combination with other data sources.
A basic analysis of inflows/outflows or a questionnaire of why people join/leave could indeed be done for a much cheaper price. This proposal can help do those sorts of analyses ongoingly in the future, but that’s just the tip of the iceberg.
The thing that we really want to look into, and it requires research not just a bit of data analysis, is the broader possibilities of what can be done when we really understand network data. Facebook and other algorithms have used some insights in these domains to make people addicted to a platform (e.g. knowing who to connect you with when you’re likely to leave) but the same nights can be used in a more pro-social way.
Some underexplored applications in the corporate world have already enabled organisations to predict which teams would perform better by looking at the patterns of communication between team members. Equally to predict which teams are likely to lose team members and even predict who’s likely to leave. Or we could think about reputation weighted voting where the reputation model is adjusted dynamically based on survey and organisational data (which makes reputation ungameable).

If we can understand those patterns in DAOs, combine them with other sources of data, and then create nudges that serve both the users and the DAOs, this is a whole realm of future of work possibilities that opens up using AI, social system design, and blockchain technology.

So we can take an incrementalist view of let’s just solve the immediate issues that Aragon has, at which point probably it just makes sense to be one small product team, or take a more expansive/disruptive view of Aragon as a platform to transform and shape the future of work and invest in research initiatives that open new possibilities (and also have some impact on current priorities, but as mentioned, that’s the tip of the iceberg) :slight_smile:
Hope that makes sense

1 Like

Thanks for your reply @daniel-ospina, appreciate it. The power of ONA is clear from your description, thanks - I’m very interested in this topic.

Also not advocating for an incrementalist approach, but rather doing the expansive approach when the time is right for AN DAO. (If the time is right is open to peoples’ different views here! My view on that is in my first comment).

1 Like

Thanks @lee0007 for bringing me in in this conversation. I particularly appreciate in this proposal the fact that the questionnaire won’t be just a matter of closed-ended questions or variables to be assigned a numerical value (quantitative evaluation), but also open-ended questions in which participants themselves can take part in generating feedback and insight (qualitative). I am curious to understand how the tool works in practice, I see that pulse offers 14-day demo version, maybe it’s possible to start from there to do an evaluation.
As it was already mentioned, there are some privacy and tool management issues that will be generated as the tool is implemented but that I think would be useful to try to answer first, such as:

  • Who will be the repository of the data? Will the metrics and the results of the survey be open-access according to the transparency culture? Or will they be transmitted to a specific team?
  • How will the tool provide an assessment of an individual person’s burnout risk status without violating their privacy? Such a warning signal will be transmitted only to the person itself?
  • This for the Aragon community: how will a possible burnout situation be managed by the community? Are we going to create an internal welfare system?

On the financial and other technical evaluations made in comparison with previous proposals, I am not commenting both because it is not my area of expertise and because of my brief membership in this community. In general, I believe that tools are useful for making informed choices when used ethically. Also, I am always in favor of research and experimentation so I can only support initiatives that go in this direction. If it were possible, in my view the ideal is always to integrate different initiatives rather than exclude them from each other (“and” instead of “or” approach).

1 Like

Jumping in late on this = my apologies…

So on the one hand i agree - the tooling side of this probably is a ways off from useful… that said thank you @daniel-ospina for the introduction to katerinabc - i looked at her github repo and was heartened to see several projects related to cleaning up authorship issues in the research space = this is still unfortunately difficult (yay orcid) however points to the type of clarity and generally thoughtful approach I can support.

Literature reviews are an interesting space and as a librarian working in a hospital we specifically do literature reviews for best practices in health care as a service for clinicians and researchers. Increasingly tools like wikidata and wikicite are being leveraged to store, reference, and contextualize information and data sets.

My .02 is that we consider something like https://docs.openalex.org/ or wikidata in the deliverables because at some point for the literature review to be an actionable product it needs broad contextualization and also needs to be editable… the underlying research in this space is still being written and it should be anticipated that “shit may change”

anyhow - congrats on your bounty = i’ll look forward to seeing how you progress

1 Like

Congratulations on the Proposal passing. It is a shame we didn’t get the Transparency on the finance of the proposal or when the funds should be released, would be great if they can be clarified now. Especially when the funds should be released. Congratulations again

Thank you @AClay

I’m sorry to hear that the reply didn’t offer the clarity you hopped for.

Attempting another response here:

  • We’re asking other communities for the same ticket item as Aragon.
  • As the timeline suggests, the work starts immediately and in parallel streams (we started with the research since Monday already) so ideally we receive the funding as soon as possible.

I hope that clarifies but please don’t hesitate to ask if any questions remain

3 Likes

@alibama @Incandenza @AcidCavin @mroldann and everyone else who commented, thank you for your recommendations for the project!

We’re hoping to engage with the Aragon community throughout the project to facilitate learning and value creation for all the project stakeholders. The best approach we have identified for that relies on creating shared spaces for discussion and learning, where we can receive your perspectives and exchange ideas.

I’ve already shared your comments with our team, and if you’d be happy to answer any clarifying questions we might have or have anything else you’d like to share so we might best orient this initiative, we’ve created this telegram group:
Telegram: Join Group Chat

2 Likes

@noturhandle Can you add wallet address to the budget section of the original proposal please for ease of reference. Thank you

1 Like