Karma weight decision making

After reviewing feedback from the first Karma discussion I am making this space to discuss the use and weighting of Karma in the Aragon DAO.

In the first session, when participants were asked what Aragon should be using Karma for the results showed: 1. A threshold for being a paid delegate, 2. A threshold for being a paid contributor.
When asked how who and how should we decide the weighting of Karma for the Aragon DAO, participants said: 1. A Forum Discussion, 2. A working group consisting of AA leadership and Guild Leads.

I have scheduled the next session for discussion in sesh for Nov 7th. The goal is to discuss Karma weighting here so we have all had a chance to think about and review opinions/ideas for Nov 7th at which time we will put together a working group who will decide the weighting of Karma at Aragon.

If you are unfamiliar with karma please check out the following:
About Karma:
https://showkarma.xyz
The Aragon Karma Dashboard:
https://www.showkarma.xyz/dao/delegates/aragon
Connecting your forum ID to Karma
https://forum.aragon.org/t/karma-reputation-system-for-contributors-link-wallet-to-forum/3810

Some questions to think about:

  • How do we represent everyones value given to the project, not just your own?
  • What is Value? Development, Saying GM, Tech support, community work?
  • What about offline value such as speaking events. Should this be represented and how?
  • Should some discord channels be weighted more than others?
  • Whatever score comes out of this will be a threshold for being a paid contributor or delegate. Do you agree with this or have an alternative idea? Should it be signaling only until a later date?

Your feedback is needed so we can accurately use Karma the way it should be used at Aragon.

2 Likes

One topic that has come up is that Karma is currently not open source. Is this something we feel is a concern at Aragon at this stage? Does this change its use case at Aragon.

Personally I don’t see a problem with it at this stage unless we start to see some extreme variations in how it is reporting. I would want to see it going open source sometime on the horizon.

1 Like

i don’t think having the openkarma product open sourced matters at all… ultimately the mashup they provide, the user interface, and then any subsequent integrations we want to add on for our convenience is on us.

on the other hand giving people a very clear sense of how these numbers are cooked seems kinda critical… like we need to be 1000% clear on that

1 Like

I love the idea of a dashboard that reports Karma (value + weighting + reward) across an entire DAO but it is a complex concept where I feel Gall’s Law applies

all complex systems that work evolved from simpler systems that worked

If we want to build a complex reward mechanism for all DAO contributors - across multiple team functions and providing different forms of value to the network - then we should build the simplest system first, and expand and improve it over time.

Given the MVP it appears, we are closer to achieving 1 than 2.

The data karma would report for governance contributors - as a subset of DAO contributors - is easier to discover, design for and learn from at this stage. If we start simple, the process of how we establish karma weightings for the governance subset could be repeated to expand to other subsets/functions/teams/guilds.

I suggest we work to achieve 2. A threshold for being a paid contributor. by designing for and learning from the process to establish 1. A threshold for being a paid delegate or an even more simplified option would be to remove the concept of payment threshold (at stage 1.0).

Agree that mechanisms to help establish transparency and shared understanding around rewards are important. However, the payment threshold introduces a complex variable to the design - quantified $ reward attached to multiple forms of weighted value

Q: Are other teams using Karma to design reward mechanisms that we could learn from?
Q: Is it necessary (needed vs nice to have) karma dashboard tied to reward at this stage ( 1.0)?

1 Like

Maybe the term should be eligibility as it is not focused on a quantified reward. On a system on 0-100, 100 being great value, maybe paid delegates would need to have a minimum karma score of 65? This would mean Karma is not the one and only deciding factor. A minimum threshold that says… you at least participate. The bar could be set very low at the start and could be adjusted up as time goes on. I’m of the opinion that if Aragon is going to publish and pay delegates… they should at least be able to meet a minimum value/participation threshold.

+1 to simple first, then expand

2 Likes

What I keep coming back to in my thoughts is the problem that we are trying to solve with Karma. I would state the problem as: measuring the value that individuals bring to the project.

Solving this problem will require accounting for different variables, there are a myriad of ways of adding value and considering any one metric will always favor some but not others.

I see Karma as an additional data point in this equation. I would suggest that Karma alone should not be the deciding factor.

Karma is great for looking at communication contribution rather than achieved contributions (say a report is delivered or a feature is implemented). It has a bias towards quantity of communication and (as far as I understand) lacks an ability to look into the content of said communication. However, it can function as a very good proxy for interaction and engagement with the the community.

If we want Karma to be more focused towards Delegates, then all governance related channels needs to have stronger weighting, as would maintaining a 100% voting record. This would imply that we are slightly changing the problem as I stated above so that now we would be trying to solve: how can we best chose delegates?

I would love to hear from others their view on the exact problem we are trying to solve here. I think that this will be really helpful to best answer the questions above :slight_smile:

2 Likes

If we want Karma to be more focused towards Delegates, then all governance related channels needs to have stronger weighting, as would maintaining a 100% voting record

Great point. Before we do the weighting we need to solidify that Karma will be one of the deciding factors in choosing a paid delegate. @fartunov @Anthony.Leuts I’d like you guys to chime in here with your opinion.

Maybe Karma becomes two algorithms with two front ends:

  1. Aimed at being a factor in choosing delegates
  2. Aimed at contributor value

I have already talked to Karma about being able to filter their algorithm, ie. remove discord channels from the results by applying a filter. This would allow me to remove channels I think are unimportant in my decision making on who to delegate my ANT too. However, that is not an option at this time.

This IMO sounds unlikely. Ultimately token-holders can choose whichever delegate they want. It’s up to them to consider or ignore different data points in making that decision.

As of today, it’s not confirmed if any delegates will be compensated.

Lastly, I don’t have a strong statement as I don’t know how the score is determined, but for me, “forum activity” is not a strong linear indicator of being fit as a delegate - some is better than none, but more does not necessarily mean better.

2 Likes

Thanks for jumping in. Of course anyone can delegate to anyone but are we going to have delegates that are chosen or advertised or listed by Aragon in anyway? If so, there should be some type of transparent threshold for being pushed out into the front. The purpose of this discussion is to determine how the score will be created, none of us know how the score is determined yet.

In any case, the only use case is not choosing delegates, thats just the one that stood out on the Karma call with the community. It would be good if you could join the next one

No clue if this will be the case.

If it is, I can only share my personal opinion: Delegates will have to make strategic asset allocation decisions - involvement in the forum and voting in the experimental AN DAO is a poor benchmark for having the necessary experience and expertise.

1 Like

In terms of Karma representing the delegate subset of contributors: diverse perspectives and the ability to communicate and provide substantiated, constructive feedback is fundamental to governance in any environment. Governance is a process or at least a collection of interrelated processes by which decisions are

  • discussed
  • made
  • implemented
  • enforced
  • re-evaluated

All of these require the ability to communicate which is what Karma currently represents in some form. @fartunov what do you believe are good measures of governance participation that can be effectively represented via the Karma dashboard?

The forum is currently the most open public therefore transparent record of resource allocation to date and benefits from the fact that anyone that cares can contribute here. Unless there is a move away from public discourse as a governance mechanism it will remain a valid data point and imo one of the most representative of AN DAO governance to date.

I think we need to remember that a liquid democracy enables people (ideally) to choose different delegates depending on the decision at hand, and for any reason they want. Personally - recognize my ANT holding is minimal but can contribute to collective decision-making - I would delegate to people with

  • technical expertise in technical decisions
  • financial expertise in treasury decisions
  • in general, to those that I see as upholding values that I believe are important to the success of DAO as an ecosystem. i.e integrity, transparency, accountability, constructive

Other people will choose for very different reasons and there’s no way to possibly provide information to inform all the variables at play. Yet information symmetry and transparency is what we could try to deliver. Karma is just one (tested and potentially effective) dataset, currently reflecting forms of governance participation in terms of time, context, and communication across a limited number of platforms.

Karma is better than the nothing we otherwise have and hopefully only one element in some form of profile that delegates could offer. I expect there will be delegates with sizeable holdings that prefer to remain anon and can achieve ends by operating through others, such is the world but in a Decentralised Autonomous Organization, we have the opportunity to provide some counter-point to the currently opaque means of governance typical of legacy organizations.

1 Like

This is a good point, specially since chances are that Delegates will be people that have not been directly involved in the project yet. We could still use karma to track their involment after they are appointed and set some requirements for an eventual second term.

Official Aragon Delegates Vs each person is a delegate only because they have a wallet. Lets make sure we are talking about the former as you can’t stop or limit the later.

This is a good point, specially since chances are that Delegates will be people that have not been directly involved in the project yet.

@Incandenza If people have not been directly involved in the project yet, they have no business being an official delegate… they wont know what the fuck is going on at all. I’d say once they have shown they at least participate in some fashion = understand what is going on somewhat, then they could be a delegate.

A poor benchmark

@fartunov What is the benchmark with all DAOs still being in an experimental phase? 1inch, Aave, Balancer, Ampleforth, Bankless, compound, Decentraland, ENS, Gitcoin, Lido, Maker and many more are doing this experiment and using it as a benchmark in some sort of fashion. I think I will reach out to some of them to see what they are using it for and how it is working out for them. Without any other alternative other than two or three people hand picking delegates, I don’t see any other competing solution that makes sense at this time.

@fartunov what would be very helpful is if you could help us determine the weighting of Karma. If you believe the forum should be only 2% of the factor in determining the weighting, please voice that. Karma might be only one of 10 factors determining an official Aragon delegate but it is a start.

2 Likes

Maybe I am doing this wrong but here is an idea of weighting for Karma. Its very high level and would need refined a ton. But it’s something to start the conversation. I think if we can get the major categories set in a percentage basis, we can then have the discussion of each one on a lower level.

  • Discourse Forum: 50%.
    ** ----- Broken down -----
    ** Proposals: 30%
    ** Other governance channels: 15%
    ** Other forum categories: 5%
  • Discord: 15%.
    ** ----- Broken down -----
    ** Guild/working team channels: %5
    ** Governance channels: %10
  • Voice Voting: 35%

Now I know Karma is not done on a % basis. It is done on a point basis but this can help us figure out importance of different areas of value.

1 Like

@brent as a matter of principle I 100% agree.

But I keep on hearing/reading that ANT holders will have complete freedom to pick whoever they want as a delegate so if this is the case they might choose someone who might have been involved but has never been visible (= would have a very low karma score).

We could still use Karma to track delegate engagement, and at that point being actively engaged could become a requirement

1 Like

One last reply that will boost my Karma while effectively decreasing my impact on this project per hour spent on it.

As a data point, Karma makes sense (not in its current form of course), as a formal requirement, it probably will do more harm than good - but again, it’s for token holders to decide.

I don’t believe there is a simple quantitative number that can serve this purpose. The data
available has a very low signal-to-noise ratio. Putting it through a formula that spits out a single number doesn’t fix that. It can be harmful if portrayed as a “strong signal”.

Not sure what data inputs should go in, but certainly, if it’s about delegates tracking, these should start after the DAO is deployed.

Currently, a few people who applied for funding through the experimental AN DAO and people (myself included) whose role was to weigh in on funding requests are given a head start. It skews incentives - contributors on proposals will have to argue who to post it in the forum. If we weigh threads more than replies, it creates an incentive to start a new thread, where replying to an existing one would be a higher net positive. The more weight we give to the score, the worse it gets.

The same applies to discord - if I pick a bounty to stir up conversations in discord, I will get an amazing score. There are strongly advantaged roles that are completely unrelated to one’s competence to be fit as a delegate.

Token holders will be the ones to determine if there will be official delegates and how those delegates will be selected - as simple as that.

Precisely

Only if token holders choose to make it a requirement (hopefully not). Most projects, to my knowledge, are just presenting it as a data point, and then tokenholders make the judgment.

2 Likes

I’ve got to agree with @fartunov on the fact that measuring these things is difficult. We had something similar happen with Sourcecred when I first joined. Since I was more visible in the community I was receiving 2-3x the amount of rewards that others were. I could see this happening with Karma as well. Those who have high visibility or have been involved with the project for a longer time will have higher scores. This doesn’t necessarily mean that they are qualified for this position. Even if we have more context of what is happening within the organization, I think we have all seen how all context can be thrown out the window in a matter of days or weeks. There is also the experience of the person that is being left out with this measuring tool as well.

This isn’t to say we don’t need some way of gauging people’s engagement with the DAO. We definitely should, at least for DAO-level or community-level delegations. But to require anyone to participate to be a delegate or for their scores to be an accurate representation of what they have to offer may be a bit much.

I also think this tool would be just generally useful to have around. For people to be able to see others’ involvement or even their own would be a cool feature and it could encourage them to get more involved if they want to (in order to get their rank up).

We just need to clarify the purpose and be realistic with who will want to be involved. Does it make it right? Maybe, maybe not. Will us pitching a fit about it change that? Absolutely not. If we want to use it, let’s use it. Then let those who want to participate and those who want to opt-out be free to do that. Then the results will speak for themselves.

2 Likes

Can you apply your weighting @brent and see how that looks? it might be easier for people to discuss actual weighting if we can see how that changes the current dashboard. Can Karma provide a list of the current items and their weighting for us to work from? (baseline)

If karma is an indicator of governance participation that is intended to help ANT holders determine who they want to delegate their tokens, then karma (as a “feature”) is serving token holders.

This conversation sounds like governance participants trying to quantify their own value rather than token holders discussing their needs.

In fact, I don’t see any significant token holders posting here or requesting this feature.

1 Like

This conversation sounds like governance participants trying to quantify their own value rather than token holders discussing their needs.

How is that constructive criticism? Is it that we are not sizeable enough token holders for your approval @evanaronson that you deride efforts to provide open feedback and communicate to reach some consensus? Pretty sure everyone here is a token holder and speaking for myself, I would like some information to inform who I delegate to.

This is an open discussion to try and demo a tool that all/any ANT holders can choose to support or not once there is a proposal. We cannot control outcomes, we can only set the initial conditions for certain qualities to emerge.

Clearly, qualities of people are emerging here, but at least some ANT holders still actually care about the Aragon Manifesto and remain committed to

A world in which every person can participate in these new organizational structures.

@brent can we exclude this category from the Karma dashboard, so the ability to have a conversation doesn’t impact “karma”?