Financial Proposal: Aragon dTech Season 2

brent · December 6, 2022, 4:08pm

At the moment that information is not accessible because this past season was split between Helpscout and Hubspot. We no longer have access to helpscout, and until we either pay for a second Hubspot or merge with the AA Hubspot, we are unable to view many statistics a paid version gives. I have been told there are some pending legal issues that have to be resolved before the two can be merged. So it is a combination of things that don’t allow us to get these statistics ATM, but that will be changing shortly.

What I can point to is the documentation and video series stats where people are getting lots of help from. We refer any people in discord to that material often and the goal is this self help information, hopefully indexed Google, will cut down on support tickets as people will find the information themselves before having to make a ticket.

Some of those stats:

In the last 3 weeks we have had 5490 views by 971 users on the documentation we have created. 8% in Japanese which is pretty interesting. 54 visitors to the tech forum, and the video series has 3,024 views.

brent · December 6, 2022, 5:52pm

How to create a DAO on Polygon is one of the biggest topics. This video we created really helps us answer that question: How to Create a DAO on Polygon with Aragon - YouTube

We don’t know the biggest challenges, only the challenges of those who contact support. There is a difference. Most common question is how they can setup their DAO on Polygon, if they can use a token they already have or have to mint a new one, and if they can use NFTs for xyz purpose in their DAO. This information is shared with the product team when we have our collaboration calls.

BearBudget · December 6, 2022, 8:49pm

Not having access to that data for November really seems like a hurdle to allowing you to run the guild efficiently, hope it gets resolved soon!

If this was tracked in previous months - sharing that data for transparency and accountability will be helpful

I am relatively new here, but why is there a separate support forum? It seems to get very little interaction, probably not worth maintaining a separate forum:

November: 4 posts in only 1 thread
October: 18 posts across 5 threads
September: 9 posts across 3 threads
August: 6 post across 3 threads

Anthony.Leuts · December 7, 2022, 7:31am

Hi Brent, thanks for the update.

A few questions: When you merged your information from Helpscout to Hubspot did you not have the information moved over? Where did all the information from Helpscout go? Usually when you merge or move data from one CRM to another you don’t lose any information. Can you please share how many tickets were received and completed per month in Helpscout before you moved over, and in your new hubspot account, how many were in there.

Thank you for sharing the other information, i.e. documentation and video series! But I think it’s important for everyone to know how many technical support tickets come in each month. Can you see if you are able to go back and get this information? I agree with Bear above that not having data is not good! According to the support forum you’ve had 37 support posts since August? Where else can data be found to add this? Thanks Brent! Let’s get to the bottom of how many total tickets you receive per month which will also probably influence the level of CRM you need to pay for when/if the team stays on its current Hubspot account or makes it over.

AClay · December 7, 2022, 8:09am

Having zero idea seems like a massive gap. It would be like finance saying we don’t know how much we spent. We must have a rough estimate. Are we not paying bounties for 24-hour support with a response time of 16 hours? If we need that 24-hour support due to the volume of tickets to meet the target hours, then it makes sense. That decision must have been data-driven. Even a rough estimate:

1 per Day
10 Per day
100 per day

Just to get an understanding, this data will be crucial as we move to the new product as well.

fartunov · December 7, 2022, 9:37am

Agree it’s a massive gap. There have been multiple requests for clarity concerning that metric.

I flagged it back in August here:

Several AA members also flagged the lack of reporting on the total number of tickets during the discussion of this proposal before voting.

The concerns about per ticket cost of the guild raised in the discussion thread were based on the observed volume of 100 tickets per month during summer 2022 (including spam messages that do not require a response).

P.S. When we don’t know the number of tickets, how are we calculating resolution time?

brickpop · December 7, 2022, 11:35am

I would also like to flag the lack of success in technical support for the important case noted above: 125k bricked in court.

This week the App team has been blocked because a Core team developer wasn’t able to work developing. He had to be addressing the125k ANT issue mentioned above instead. Which he was able to solve.

The App team is super tight on schedule and losing key developers just because “support” cannot provide support is (at the very least) concerning.

Finally I would also like to flag the “Discord user who stopped responding”. The user definitely responded to our core team developer.

MathiasAragon · December 7, 2022, 1:27pm

Hi guys and thanks for the report.

I have to ask also for the number of support tickets. That is crucial information and could also be obtained in a more manual fashion (counting the number of tickets in the table).

Also, I would like to know what are the processes in Mas. I saw on the linked notion page (Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.) that 35 out of 42 are not “Completed” and 0 are currently in “Being Fixed” state.
So either there is no work being done or it got migrated somewhere and the linked Notion page isn’t up to date.
Either way, a clarification on how the Mas project operates would be nice.

Secondly the locked funds in Court:
Are these the funds that we started unlocking yesterday? I had a call with the user and Pieter from dTech and we are in the process of unlocking them.
The user has all the necessary information he needs to continue today with the withdrawal by himself.

SpectraCryptoVerse · December 8, 2022, 8:31am

Hey Guys,

Thank you for your questions. We are in the process of answering them to the best of our ability. I am travelling today but will try to post it today or tomorrow.

Shawncubbedge · December 8, 2022, 3:45pm

Thanks for the work you guys do. It’s unfortunate this switch has caused issues in reporting. Although it was a necessary change to ensure cohesion within the org. I trust you all will do your best to provide a report as soon as possible.

In regards to the “bricked” 125k, id be interested to know how many issues needed to be elevated/addressed by the core team. If this is a regular occurrence then this is definitely an issue. However, if this is the only issue in the past 12 months (since dTechs inception) to be forwarded to the core team then I would say good job. Im not a part of this workstream so I may be mistaken, but if 125k worth of user funds were at risk I would want a second opinion as well.

Regardless, I’m keen to see the final report so that your work can be represented properly. Feel free to reach out if you want some help

lee0007 · December 9, 2022, 3:39am

There’s a pack [of] here worried about the number of support tickets yet the elephant in the room is that dTech is frontline on efforts to retain users on this network. They are not the developers of the technology.

You all might want to consider offering a little more respect to the team of committed intelligent people in dTech that are all too well aware of the extent of failures of the Aragon product (and maintenance), which they work to resolve and support 24/7

As a matter of public transparency

what failure causes 125k to be bricked in Court?
who on the development side is accountable for this failure?
shall we also discuss the other losses that Aragon DAO users have experienced in the past season?

The people accountable and the organisation responsible are NOT yet members of a DAO. There’s a growing legal liability here. The ongoing and repeated lack of accountability evidenced throughout this forum is imo grounds for an investigation of professional negligence (at best)

.

Joan_Arus · December 9, 2022, 10:22am

Hi Renee,

I’m quite disappointed to read your message. I can understand differences of opinion, even straight and plain opposition to criticism. However, I can not understand threats.

You are correct in saying that the people accountable and the organization responsible are NOT yet members of a DAO, but as matter of fact, neither are you nor anybody else, as the Aragon DAO is yet to be launched. As voted by ANT token holders, Aragon Network DAO Charter will be terminated 15 days after enacting the new Aragon DAO Charter. A new Aragon DAO, with a new Charter that will be voted during December 2022, will be deployed.

AA teams have been working relentlessly in: a) Designing the tech deployment of a new DAO with delegate voting system that is safe to deploy, in coordination with Blossom, b) Due diligence on the legal path forward, c) Prepare the teams to transition towards the new structure, and d) executing the strategy published here, with the first steps being the release the new aragonOS (already on testnet) and the new Aragon App (on mainnet Feb-23). So, quite a bit. Some team members have double / triple Roles, as they engage both in their regular workstream and also in the Aragon DAO Transition. I don’t need to enter in evaluating their professionality, results will speak by themselves. I’ve never worked with such a professional and high-performing team.

Whenever you refer to “growing legal liability”, please be explicit towards who and the cause for that liability. Otherwise, I’d ask you not to engage with threats. We’ve got enough of those as a project, don’t you think?

The core teams at AA are perfectly aware of the limitations of the current Aragon product offering, and in fact, it is the acknowledgment of those limitations that lead to our new product strategy. See also Course of action here for more context. If dTech detects a bug in the frontend or something that doesn’t work properly in the course of their work of MaaS, they should open a support ticket so it can be solved (AFAIK that’s one of the main functions of dTech and why the guild initially was created for).

Once the heavy work regarding the steps abovementioned has been done, it will be subject to Forum discussion and after that, token holder vote. These steps can not happen whenever you want. These steps can only happen once they are completed, full stop. And meanwhile, I believe that there has been a constant and steady flow of communications to all community members via the weekly Eagle and several forum posts. Could I have been more responsive in replying comments in the Forum? Indeed! But I’m limited to the lays of physics. I can’t extend the length of day. Therefore I must prioritise my attention to the responsibilities and objectives that I have for the quarter / year. Can this be cause for “legal liability”? I doubt it.

Fortunately, that work is almost complete. As of yesterday, the smart contracts of the Delay App + TAO Voting deployment were finalized, so we can start the audit. During this month of December we will put forward the new Aragon DAO Charter for vote, so token holders can express their support / rejection regarding the proposed path forward. The vote should be set in a way that token holders can have granularity in their signalling too.

Lastly, I’m a strong believer of exit optionality. If you don’t like how something is working and don’t feel aligned any more with the Governance Framework, you are free to exit whenever you want. You can always sell your ANT and join a community that is more aligned with your values. This may be a more productive use of your time other than engaging in empty legal threats and accusations of professional negligence (which IMO could be considered cause for a permanent ban), and for sure, a way more productive use of my time (or anybody else).

SpectraCryptoVerse · December 9, 2022, 1:08pm

Thank you for your patience, we wanted to make this response a guild effort. Please find the replies below, hopefully this will address everyone’s concerns.

@AClay, @Ricktik6

We always believed that the number of tickets is not really as important as whether users can get relevant information. The better the documentation and instant help (#bugs-n-feedback and #questions channel on Discord) the less tickets we are going to get as people only raise a ticket if relevant information is missing or hard to find. We cannot do a lousy job on documentation or Discord just to keep ticket numbers up. The number of tickets in November was 107 but the new ticketbot feature in HubSpot is already reducing the number of tickets. The stats we were trying to get out of HubSpot was how much the new ticketbot reduced the number of support tickets as it gives quick and relevant information to the most frequently asked questions. When I was able to look at the stats (until the trial ran out) it was roughly 3:1 so without the ticketbot the number of tickets could be as much as three times higher. Additionally with maintenance and fine tuning we will hopefully reduce it even more. HubSpot (in the free version) does not give us additional info on the KPIs in our proposal i.e. first response time and resolution time and it works differently. We are still evaluating it (so we can adjust our OKRs and KPIs accordingly) but we ran into a roadblock with the trial version expiring and not being able to integrate into the AA HubSpot. In November the ‘Ticket average time to close’ measurement from HubSpot is 20,7 hours which is within the 24 hours in our proposal.

Predicting the number of user support tickets for the new App is somewhat impossible as we are not involved in the user/developer documentation creation so we do not know if relevant information will be available and easy to reach.

All this season:

Most active channel at Aragon is the Questions channel

dTech Voice Hours

Messages in the questions channel

Messages in dtech-general:

Most active channel at Aragon is the Questions channel

@Anthony.Leuts

HelpScout went down after requesting payment fell on deaf ears and even if we wanted to we could not have paid the current subscription out of dTech’s budget due to the payment setup at HelpScout. So after a few days waiting for AA to sort out the payment we flip the switch to Hubspot which was cca. 10% setup. As discussed numerous times before using HubSpot and/or migrating to it is NOT in our proposal as a deliverable nor was it in our previous proposals. dTech has never had an ‘official’ approach to setup Hubpot.

We pretty much had to learn HubSpot and set everything up from scratch, replace the floating widgets on the website etc. We did not migrate over, we were forced to move as at the time we did not (and at the time of writing still do not) have access to HelpScout after its closing and revocation of access due to non payment. I am assuming that once the payment issue is resolved we will have access to all historical data. The question is whether it is worth paying another year’s worth of subscription for this data.

Also, as far as I could tell data could not have been moved over as HelpScout is not a CRM, simply a support ticketing solution.

This abrupt move to Hubspot was talked about in the AN DAO all hands meeting numerous times. Currently there are Hubspot tickets, Discord, the forum, and other smaller platforms such as telegram and the tech support forum. Most support happens in Discord in the Questions channel which is the most active channel on Discord, the dtech-general channel, Bug-n-Feedback, and sometimes spread out in other channels and DMs and groups that we are added to.

@BearBudget,

You can find ALL our End of Month and End of Season reports on our Notion page here under Operations → Transparency → Proposals, reports.

I agree having another forum is not ideal. This discussion has taken place multiple times already with some people concerned with posters looking for support not reading and just posting in governance sections. The purpose of the forum is to hopefully get issues indexed into Google so people can resolve their problems and further reduce tickets and provide a better user experience.

@Brickpop,

I don’t think it’s fair to say a user getting their 125k ANT is a measurement of success for technical support. We escalated the issue as it was a large sum of money and the Court smart contract error suggested a bug. I have confirmed since the EOS report Barukimang has been in contact with him as well as a few other people here so that was some miscommunication on our part. I don’t think there were any other instances over the last year where support has needed the time of core app developers. Before there was technical support, a lot of the core team’s time was taken up dealing with support issues, this was over a year ago before dTech existed. For this particular issue concerning the 125k bricked in Aragon Court, Mathias provided support for around 30 minutes so I don’t think that was the cause of your block.

@fartunov

As discussed before, dTech is doing way more than just tickets and measuring the success of the guild is not based solely on tickets. In Season 1 we have tried to provide additional metrics we thought were useful from both HelpScout and Discord to show more data and dTech’s activity but it was seen as “unnecessary” and out of scope so we stopped in Season 2. You can find ALL our End of Month and End of Season reports on our Notion page here under Operations → Transparency → Proposals, reports. All four End of Month reports in Season 1 include detailed metrics to which I linked at the end of the EOM report here on the forum.

fartunov · December 9, 2022, 1:25pm

Thanks for your response, bud!
The old reports included random stuff like # of Discord messages which I agree is vanity and unneeded.

Though the reports don’t zoom in on this one primary metric which no one ever said is “unnecessary”. On the contrary, there are people explicitly asking for details about it - i.e., how much spam (e.g., “cross-promotion”) are we getting vs. legit technical support requests, and what are common requests?

For this season, how do you track resolution time if you don’t know the number of tickets? It’s one simple metric and arguably the most important one for dTech.

I get why it would not be easy to provide if the question was “what is the TVL impact of faster support.” which would be some hand-wavy assumptions-ridden “calculation.”, but the question is pretty clear cut.
The number of support requests and, for example, how they map to the defined scope. Without that context, resolution time is a vanity metric

If resolution times go up because you help someone retrieve XX thousand dollars, we are solving real problems.
If resolution time goes down because all the requests are for things that only require a “not interested” response and/or are out of scope, do we need around-the-clock support?

If you believe that request (which has remained outstanding since August) is unjustified, I would be curious to understand why?

Anthony.Leuts · December 9, 2022, 3:25pm

Hi @SpectraCryptoVerse

Thanks for the response!

Please tell me if I am wrong/confused, but in the proposal on August 7 it specifically says Hubspot? See below screenshot:

If it was in fact in the proposal which I see, is there a reason it took until November to begin implementing? Let me know if I am mistaken.

Thank you!
Anthony

lee0007 · December 9, 2022, 7:26pm

Suggesting the that there are no members of the AN DAO a mechanism through which raised capital was distributed this past year by the legal stewards futher underscores my questioning accountability and duty of care. As I am not subject to an NDA I understand why you wish I quietly exit. If I didn’t care about the success of this project, I wouldn’t be here

To be explicit I refer to product users in S2 that have lost funds through interacting with the product. As a foreseeable risk of not maintaining a promoted product imo they have legal recourse. Not a threat, just the way duty of care works.

My suggestion was to show some respect to the frontline members of this team. Support issues and ticket numbers are first and foremost a failure of the product and product maintenance, not the team who have rallied to support product users.

Joan_Arus · December 12, 2022, 10:28am

As I am not subject to an NDA I understand why you wish I quietly exit

Very disappointed to read this. The way you write seems to imply that I have some kind of concern for you “disclosing” sensitive, dark and evil information. I really don’t care whether you want to exit quietly or loudly, with or without NDA. Do as you wish, really. You’ve lost all credibility to me.

All I ask is for you to simply stop bullying AA team members, or making empty legal threats. You have a strange way of showing respect by attacking others, and this is not the first time you are openly hostile against AA team members in public or private spaces.

FYI: dTech’s main initiative is precisely to provide Maintenance as a Service for legacy and new products. That’s why the guild was funded in the first place. So not really getting your argument here.

MathiasAragon · December 12, 2022, 11:13am

@SpectraCryptoVerse many thanks for your detailed report and answers to the questions we have raised.
I agree with you and others that numbers like “messages in #questions” or “hours spent in dTech voice channel” aren’t a help at all.
Would it be possible to track the support requests in these channels for the next season? A simple Google form would do that the support fills out once a request is completed.
This way you and we have more accurate numbers on support requests aka tickets for the next report.

As asked in my previous post. Could I get a better look into MaS and what work is currently being done there? Because in notion there are 0 tasks in progress.

When I checkout the MaS project in Jira I see some tasks in progress (Goerli changes that are unassigned are currently on my to-do list as a DevOps engineer). And 1/3 of all tasks were done or are being done by core members (not including the 3 “add goerli support …” that are unassigned but in progress).

@Shawncubbedge

If this is a regular occurrence then this is definitely an issue. However, if this is the only issue in the past 12 months (since dTechs inception) to be forwarded to the core team then I would say good job.

I get on average around every 2-3 months requests from dTech (mainly @barukimang ) to resolve issues mostly in Client. Idk if it is my role as DevOps or as Core Dev.

barukimang · December 12, 2022, 11:48am

Hej Mathias,

Saw that your questions on Mas Software Support had not been fully answered yet, so here some more clarification:

Question: What are the processes in Mas?

The general Mas process is described in the third heading on this page:

For product bugs we have a process for Aragon Legacy Products and another process for the new Aragon App:

Process Aragon Legacy Product bugs has 4 statuses:

Discovered: can be by a user reporting it to us through Discord or Hubspot or dTech members finding bugs themselves
Being fixed: if a bug has a severity status >= 4 then we look into fixing the issue immediately ourselves by creating a PR. If we lack the technical knowhow to fix the issue we escalate the issue to the ‘repo owner’ using this Jira Board. Note: some high priority issues are immediately reported at the Jira Board
Fixed!: issues which have been fixed successfully
Outdated: discovered bugs which were never deemed important enough to reserve precious time for to resolve

dTech members have started self-training on use of the new App as part of our Launch Ready Plan. Also dTech is doing direct QA for the App Squad with 2 complimentary processes:

Process bugs found during self-training (all credit to @Carla78 who did a lot of work on this!):

Report: describe bug in New App Bugs database (scroll down on this page to see the database)
Confirm: another team member will test and confirm the Bug
Forward: if the bug is critical, Pieter creates a ticket at the App Squad Jira Board, and assigns it to Dickson
Feedback: Pieter changes bug resolved status to ‘Yes’ if bug fixed

Process bugs found during QA:

Report: create a ticket at the App Squad Jira Board and assign it to Dickson
QA: when ticket reaches QA column, tests whether bug has been resolved

Question: So either there is no work being done or it got migrated somewhere and the linked Notion page isn’t up to date?

You are right the database for ‘legacy bugs’ was not up to date! I updated it:

Discovered: 1/43
Being fixed: 0/43
Fixed!: 11/43
Outdated: 32/43

Hope this sheds some , and feedback on how we can improve is wlcm anytime

barukimang · December 12, 2022, 12:08pm

Ah seeing this new message now

Question: Would it be possible to track the support requests in these channels for the next season?
Think @SpectraCryptoVerse can best answer this

Question: Could I get a better look into MaS and what work is currently being done there?
See my other response just now for details. You are right, the only issues in progress at the moment are the ones which have been directly reported on the Jira Board.

Now Ramon is gone, you are the only person I know off in Aragon that could solve some of the issues with legacy products. I know how busy you are, and that it is maybe not even your direct role, so that is why you see issues at the Jira board unassigned (am scared to assign them to you tbh haha).

So if you can and are willing to do these tickets, great! If not then I think we should perhaps get another person with the skills to do them since maintenance on Client will still be needed for the time being no? Wdyt?