Court Sim Analysis
This post will serve as an introduction to our effort to simulate the economic mechanism used to organize jurors in a decentralized court. You can find the code on github.
This model is relatively naive, it makes several assumptions about juror behavior in order to create a baseline model for analysis. It is important to think critically about these assumptions because no model will accurately reflect the real world, but by carefully constructing a model we can use the model to better understand and reason about practical applications and mechanism design.
- We assume jurors act honestly
We make the assumption in this model that jurors act honestly. In reality we are very much concerned about how the mechanism performs under the assumption that jurors are profit maximizers who may attempt to take bribes or make credible commitments in order to gain an advantage over other jurors. However, by assuming they act honestly we can analyze a baseline case under an idealize set of assumptions, before incorporating more complex agent behaviors.
Additionally, we expect that in the event that malicious juror participation leads to a result that is bad a futarchy inspired fork choice rule could be implemented as an effective way to isolate honest behavior.
- We assume disputes have binary outcomes
We make the assumption in this model that each dispute has only two outcomes and that one outcome is correct. In practice there may be many more remedies for disputes than just two, however, there are reasonable ways to handle disputes such that jurors are only presented with two options. For example each party could submit their desired remedy, and jurors would select only between those two options which was more fair. So while it would be nice if the mechanism works for non-binary (linear or non-linear), a mechanism that works to resolve binary outcomes can be considered minimally viable.
- We assume individual jurors have imperfect but normally distributed estimations of correct outcomes
For each dispute each individual juror gets a belief about the correct value of the dispute based on an normal distribution. This means that even though jurors always act honestly, jurors may vote for the incorrect outcome. This seems like a reasonable way to model the subjective nature of disputes.
- We assume evidence and estimations do not change with appeals
For each dispute we select an initial cohort of tokens held by jurors, if necessary we expand that selection to include more tokens, but all jurors assess the dispute initially and do not update their beliefs until the dispute (as well as any appeals) is resolved. In reality jurors may change their opinion through the appeals process, but its not clear what (if any) additional conclusions could be drawn by adding that to the model.
- We assume that for each dispute if the incorrect result is returned, it will be appealed until it is no longer possible to appeal.
This feels like a reasonable assumption for the purpose of the model. In reality there may be cases where the result is incorrect but close enough that it does not make sense to appeal further, but such a result should arguably be considered a success. If the result is not close then the user should be confident that continued appeals will be profitable, and if the user doesn’t have the cash to pay for appeal fees they could request an underwriter.
- We assume tokens are non-transferrable outside of the mechanism
Jurors are initialized with a certain set of tokens and lose or gain tokens through participation. There is no way within the model for them to buy or sell tokens, and so token in the model act like reputation as opposed to a staking token.
- We assume that jurors have sufficient attention to participate in all disputes they are selected for.
Jurors are randomly selected proportional to their token balance to participate as a juror in a given dispute, having a larger proportion of tokens increases your chance (and therefore frequency) of being selected as juror. In reality there is an upper limit to an individual juror ability to adjudicate disputes. This limitation of the ability of jurors to participate as individuals is currently not modeled.
Using the default parameterization of the model:
- Dispensation Percentage = 0.1
- Belief Threshold = 1 (standard deviation)
- Number of tokens in supply = 40
- Number of jurors = 20
After approximately approximately 650 steps the model produces the following results.
The number of successful disputes seems to be quite high, but the ratio between successes and failures declines as the model progresses. Tracking the gini index of the token distribution shows the token distribution getting more and more centralized over time, which offers a clear explanation for the declining result. By the end only a single juror is participating, and the result depends entirely on an individual estimation of the result.
The combination of number of jurors, number of tokens, and the dispensation percentage all impact how quickly the model converges to a single juror holding all active tokens. So with a large number of jurors and large number of tokens relative to the dispensation percentage the time it takes to centralize can be made longer, but regardless of the parameterization the trend over time is the same.
This suggests a couple of options.
- If tokens were made transferrable and we change our assumption about jurors ability to participate in all disputes to introduce an upper limit/cost to participating in the adjudication of additional disputes we prevent the accumulation/centralization of tokens in a small subset of jurors over time thus stabilizing with a more decentralized distribution of authority and a higher degree of successfully resolved disputes.
- We could keep the non-transferrable aspect of tokens, but introduce an upper limit/cost to participation with a large number of tokens, and introduce a decay mechanism where token balances as a percentage of the supply decrease over time.
- We can introduce a fork choice rule if results are appealed to the point where all active tokens are committed. This would split the court into two versions, with the Aragon Network chosing one or the other depending on a Futarchy market based on a prediction of the amount of future fees collected by each version of the network. If we can assume that the market will favor the “honest” court in these instances a concentration of tokens may result in a fork, but we may not have any failed disputes.
The first and second options are similar in intent but approach the issue differently, however the first option seems both simpler and also more beneficial to jurors which actively and successfully participate. The main benefit of non-transferrable tokens as described in the whitepaper has to do with making it more difficult to acquire tokens and more difficult to exit after being malicious, however, there are staking mechanisms (eg via a bonding curve) that can also address these concerns. The third option is independent of the other two, but is difficult to model and would be too inefficient to be relied on in all but rare instances.
As a next step I will be working on adjusting this model to reflect the assumption changes implied by the first option. It also seems helpful add an efficiency metric that makes it easier to see how frequently appeals are neccessary,