SVC Ready [Beginner]

    Keeping Terra data and data products up to speed is a full-time job. There are many issues faced, some of which include: paying maintenance costs (in both time and money), query refreshes and broken functionality, etc. This analysis will be conducted in 2 parts: 1. Add any items to the above list that you think are important to help maintain your best submissions as an ongoing resource. Anything goes here: it could be “get more feedback on my answers”, “bug bounties”, “regular community calls to review”, “create an Observer squad to monitor”, “improved ticketing system”...tell us what’s important to ensuring that your best work is accurate, fresh, and up-to-date 2. Propose a solution that involves leveraging community resources (that’s all of us!) to help monitor and maintain high-value dashboards, like the Acme of Skill submissions for Question 159. Be clear about the 5 W’s (who, what, where, when, why).

    Fixing errors that might (and will) arise

    • Broken functionality: the bug-bounty system could be used here, where the user spots out a broken functionality and is rewarded for it accordingly. Could also introduce certain broken functionalities to train users to spot them (i.e. create intentional bugs so that users can practice spotting/fixing them so that then they can be able to spot them/fix them when they won't be intentional).

    • Wrong logical analysis (SQL): Bring in more data scientists as well as data engineers, who would build new models, such as NLP models that would only accept submitted code that is of a certain type and logic. For example, build models that would accept SQL code only of a certain length or only if it includes a specific contract. Note: these models would have to be trained properly so that there is no discrimination between submissions.

    • Content that might be outdated: can be avoided by query re-runs, to ensure that all data is up to the current date.

    Maintanence costs

    • Monetary costs: any infrastructure (AWS, Azure, GCP, or a local infrastructure) can only handle so much. What has to be kept in mind is that users could be spamming an infrastructure with many queries, and this has to be avoided in order to avoid high costs. One way to avoid this is to deploy a more flexible scaling up/down of the system, which would let containers/EC2s scale up dynamically according to user activity hourly/daily.

    • Time-consuming aspect: in order to grow the community and ensure a high number of bounties distributed, it is difficult to keep the time aspect as low as possible. The number of bounties available should increase with time if the growth of the community is to be maximised. The way to minimise the time spent would be to have some sort of NLP model as mentioned above that would check the submissions automatically and reject the ones that are not to a high standard. If this automated approach is not plausible, then more people could be hired to validate the submissions, whether it is on a contract basis or only in times of high demand (i.e. hire an extra person to check submissions only for one day a week, ensuring that this is still within the budget).

    Maintaining submissions of high quality

    • Introduce small bounties which would be for updating/improving the high quality Dashboards (these could be done fairly rarely, for example, every 3 to 4 weeks). This would bring motivation to the community, as they would be able to get hands-on experience with the dashboard. It is also a win-win situation, as Flipside gets continuously maintained, QAed dashboards and the users that do the work get rewarded for doing so.

    Maintaining motivation for all users

    • Maintain or increase if possible Bug Bounties. This is to ensure that the community has enough projects to work through to improve their skills, as well as help, maintain Flipside and provide new insights about the data. What has to be kept in mind is that not all questions will be interesting to all users, and not all questions will be accessible to everyone. For example, someone might be at the Advanced level, yet not have the skills to solve a particular bounty, hence it is best if there are different bounties available for different skill levels.

    • Provide comments when the submission is rejected. This can be time-consuming, however, it might be very demotivating for users to get their submissions rejected with no comment. Providing feedback, even if it's just a short comment would ensure that the user knows what they did wrong so that they can improve in the future and avoid being rejected again.