Aragon Govern and Court Exploit Test Report

TL;DR

On September 3, 2022, two individuals decided to test a simple “no code” exploit on Aragon products, namely Aragon Govern and Aragon Court. Specifically, the test was conducted on the infrastructure used by the Aragon Network DAO - the biggest (in terms of treasury and number of transactions) organization using Aragon Govern. This report outlines the rationale and findings of that experiment.

Rationale

In October 2021, Aragon launched the Aragon Network DAO (AN DAO). AN DAO was designed around optimistic governance and utilized two Aragon products - Aragon Govern (optimistic governance framework) and Aragon Court (dispute resolution system). A main reason for choosing this framework was to test and dogfood Aragon’s products.

Everyone involved in the AN DAO has learned a lot about the social layer of a DAO through that experience. However, the on-chain implementation of AN DAO was never battle-tested. Since the focus of Aragon is building tools that allow everyone to experiment with governance at the speed of software, leaving the Aragon Govern and Aragon Court products untested would be a missed opportunity to draw learnings for future products and deployment of future DAOs.

The system elements that had not yet been tested in AN DAO were :

  • Aragon Govern
    • Is Govern functioning as intended?
    • Who is watching in a governance set-up that relies on “everyone watching”?
    • Is a 5-day delay window sufficient to catch and prevent a malicious action that is obscured on the social layer as a transaction with a believable justification?
    • Do users (in this case, DAO members) know how to dispute a transaction?
  • Aragon Court
    • Is Court functioning as intended?
    • Are Guardians well trained to use the system (note : although Aragon conducted a “Guardians” training with some test disputes on Court, such training was heavily communicated to ensure Guardians would be aware, which is not the expected behavior in real-world conditions)?
    • Are Guardians still alert for disputes?
    • Is the current fee model in Court enough to get Guardians to participate (note: some active token holders have preferred to unstake their ANT, in essence, opting out of their role as Guardians to be able to vote on AN DAO governance decisions)?
  • Overall
    • Is the combined system Govern + Court (in its instance as used by AN DAO) secure?
    • Are current teams suited to assist if either infrastructure above does not operate as intended?

In that context, @ramon and @fartunov (both at the time were contributing to Aragon as members of the Aragon Association, and @fartunov was also a member of the Executive Sub-DAO within the AN DAO) set out to run an exploit test on the infrastructure. No one in the Aragon Association or the AN DAO was informed ahead of the test, in order for it to be effective. Certain individuals were informed during the test, to avoid any harm to parties directly or indirectly involved. The test hypothesis was simple: “Can a believable transaction successfully exploit Aragon Govern and Aragon Court and withdraw funds from the AN DAO treasury?”.

Timeline

  • September 3rd, 2022
    • New wallet is created (0x75e529F523E41623F45d794eF1C023caF6E2d295)
      • Wallet is funded with ANT (sent by Ivan)
      • Wallet is funded with ETH (sent by Ramon)
    • Transaction is scheduled in Govern, with a fake (but believable) justification.
      • Justification is modeled after that of previously executed transactions. Specifically, it is presented as a refund from the main DAO to a sub-DAO (the Executive Sub DAO) for a fictitious payment to an external vendor (RnDAO).
      • The execution would transfer 12000 USDC and 1800 ANT to the scheduling wallet.
  • September 5th, 2022
    • Rick noticed the transaction (due to his role, he was able to quickly assess that the transaction is fake)
    • The transaction is flagged only after Alex confirms its illegitimacy and the DAO members realize it is an exploit.
    • Alex drafts a rebuttal based on the charter alongside a number of other DAO members
  • September 6th, 2022
    • Alex requested help from dTech and Tech committee because he couldn’t challenge the transaction on Govern UI
    • Alex managed to figure out the issue by himself and challenge the transaction.
    • To do the challenge, Court terms need to be up to date. Tech committee member alerted about this, and Alex managed to update it.
    • Dispute is created by Alex.
  • September 7th
    • Voting period begins
    • It is noticed that the dispute does not show proper information on Aragon Court (description, original justification, or dispute reason). Alex requested help from Ramon and Mathias.
    • Mathias created a PR to solve the problem. After testing in staging, the fix is deployed to production. Information is now available.
  • September 8th
    • No guardians cast any votes. The dispute went into the reveal stage (although there were no votes to be revealed)
  • September 10th
    • After the reveal stage ended, the dispute outcome was Guardian refused to vote (as expected), and the dispute entered into an Appeal decision (meaning, it is open to be appealed)
  • September 12th
    • The appeal window ended - The dispute is ruled against the original transaction (the bad actor), given no guardian voted
  • September 13th
    • Ramon enacted the decision and settled penalties (guardians got their ANT slashed because they didn’t vote)
    • Transaction on Govern is available to be resolved (meaning, finish the challenge process)
    • Ramon resolved the transaction, it is ruled negatively, meaning it was denied and stopped.

Test Results

A malicious transaction did not manage to take funds out of the treasury. Looking at the specific tested elements:

  • Aragon Govern
    • While there are UX/UI challenges, Govern functions as intended
    • Within the first 48 hours, a single DAO member noticed the transaction, however, the 5-day delay proved sufficient
    • Dispute user flow presents challenges but is functional
  • Aragon Court
    • Both contract and UX/UI presented challenges, but those were overcome within 24 hours.
    • No guardians voted on the dispute.
    • As Court’s default action is to block the disputed transaction, the malicious transaction failed. Further implications are discussed in the Learnings section below.
  • Overall
    • The combined system Client + Court in the configuration used by AN DAO and the social layer of current contributors have successfully prevented a malicious transaction.
    • While the teams managed to overcome all technical difficulties with the dispute process, the overall familiarity with the system is low

Learnings

The idea of court and optimistic governance is still viable, but most likely as optional (and fully customizable) plugins rather than standalone products.

User experience - multiple elements of the user experience of the system contributed to increasing the vulnerability of the system and made it more challenging for the DAO members to react to the attack.

  • There are no notifications for scheduled transactions, although we do have EPNS pushes enabled - the teams were unable to determine why notifications (up to our knowledge) did not warn any DAO member for the transaction.
  • An asynchronous user journey spanning multiple apps without clear user flow is not accessible and makes execution challenging and mistake-prone
  • The information structure is poor, with important pieces of data scattered sporadically. For example, when a transaction is disputed, that information is available at the very bottom of the page, often outside the default view screen.
  • Even in the user interface, the displayed information is developer-friendly instead of end-user-friendly and relies on knowledge of specific technical terminology.
  • In Court, there is no clear flow separation between users who are setting up or acting on disputes and the guardians.
  • Dispute fields (description, evidence, justifications, etc) are not clearly presented. For example, the justification of a dispute is called «Dispute Evidence 1» with a linked PDF (labeled: DATA).

Court mechanics - the overall economics of Court is currently not sustainable and creates vulnerabilities for any system relying on Court:

  • Cost of operations and disputes is high
    • Given the high dispute cost, a malicious actor can use automation to drain the treasury by scheduling many small withdrawals. If the cost of a dispute exceeds the amount of each withdrawal, disputing the transaction is not economically justified for the DAO
    • In addition to the cost of the dispute itself, during the test, Alex had to supply ETH to Court’s heartbeat contract, which significantly increased the total expenses related to blocking a malicious transaction. No one party has enough economic incentive to keep Court’s heartbeat running, given current usage.
  • Incentives for Guardians are not sufficient at current Court utilization levels, and no guardians turn up to adjudicate in the allocated time window.
    • Having a default of “block disputed transaction” provides some rudimentary security against attacks on the treasury when no Guardians take action. A malicious actor can, however, use that mechanism to dispute any action by a DAO, in essence compromising the censorship resistance of that DAO.
    • Given the low number of total and active Guardians, an exploiter could also be a guardian and appeal the dispute until they are drafted and be the single Guardian voting on the dispute.

User and team preparedness - AN DAO contributors (as the representative sample of DAO members) demonstrated a well-functioning social layer. The DAO quickly identified the malicious transaction (without assuming ill-intent from any actors leveraged to make the transaction believable) and appointed a person to take action on-chain. The largest social layer caveat was that a single individual was tracking scheduled transactions. That being said, the need for more accessible and complete documentation and guides for both technical teams and end users is apparent:

  • Even experienced DAO contributors have large gaps in their ability to use the system. Determining the correct next step was subject to “learning on the go” rather than pursuing a deliberate and informed course of action.
  • Current technical teams did not have enough in-depth context on the workings of the products to quickly and confidently resolve issues as those appeared:
    • The teams did not manage to identify that disputing the malicious transaction requires the disputing wallet to have a sufficient DAI balance.
    • The teams were not aware that the default mode of Court is to block a disputed transaction, even if no Guardians show up to vote.

Conclusion

The test was deemed a success by the individuals leading it, as it provided a lot of previously lacking insight into the functioning of a Client + Court system. The Client + Court system (at least in its specific configuration used by AN DAO and including AN DAO’s social layer) -withstood the tested no-code exploits in the form of a malicious transaction. Taking the learnings outlined above a step further, we would conclude that optimistic governance is best suited for low-value high-trust environments (i.e., small payments from a sub-dao wallet with a spending cap. This said, resorting to court for small transactions is not economically viable. Furthermore, in its current state, Court is not suited for securing large treasuries. The individuals hope the test provides interesting insights that will help the product team/s working on governance infrastructure at Aragon and beyond.

The individuals leading the test, @ramon and @fartunov, would like to apologize to the AN DAO, Aragon Association, ESD, and RnDAO teams for any inconvenience or distress running this test might have caused. For this to be a valid test, it was important that no one was informed unless it was strictly necessary to avoid harm against any parties directly or indirectly involved. We sincerely believe that the product learnings are of great benefit to Aragon and the ecosystem at large. We hope they will be applied to inform better products and governance models in the future. We remain available to answer any questions you may have.

9 Likes

This is so interesting. Lots of key takeaways in this report.

Thank you for conducting this experiment @fartunov & @ramon.
Kudos to @Ricktik6 and @AClay for spotting this and taking action!

4 Likes

Nice one, @fartunov !

1 Like

Great test and thanks for sharing @fartunov @ramon!

Pasting a dashboard for Aragon Court in case anyone is interested to know # guardians and $ANT staked

5 Likes

Thanks for sharing the dashboard transparency!
Question on the table showing Disputes by Month: Using September 2022 as the reference does two days represent a measure of

  1. Time: # days disputes were available for Guardians to vote &/or
  2. Time: # days taken to resolve the dispute &/or
  3. Disputes: # of disputes September 2022

It’s Disputes: # of disputes, legend was corrected to Monthly disputes.
Thanks for pointing it out!

1 Like

Great work! This adverserial testing should happen much more often.

2 Likes

Hello, this is excellent see, however, have any adjustments been made since to better secure the process from some of the findings?

Hey, as far as I understand, the AN DAO is being wound down (as per a tokenholder signaling proposal), and the team designing the infrastructure of Aragon DAO is aware of the shortcomings of the old model.

Furthermore, likely, the products that carry some of the vulnerability (i.e. Govern) will be reworked or discontinued.

1 Like