Create an Eval for Your Agents

Event: A free, online, individual hackathon where experts build evals that show what frontier AI models still can't do

Platform: HackerRank event page: hackerrank.com/event/micro1-eval-hackathon

Schedule: Challenge opens Fri, Oct 23, 2026 at 12:00 PM PT. Submissions close Mon, Nov 2, 2026 at 9:00 AM PT

Questions: hackathon@micro1.ai

About the Event

1. What is the micro1 Eval Hackathon?

A free, online, individual hackathon where you use your expertise to find something today's frontier AI models still can't do, and turn it into a rigorous, reproducible eval. It's not another "build an agent" hackathon. The core skill is discovering real model limitations and measuring them well.

2. What is an eval?

An eval (short for evaluation) is a set of tasks, plus a way to grade them, that tests how well an AI model performs. A good eval shows clearly where models succeed and where they fail. In this hackathon, you build one from your own domain knowledge.

Eligibility and Registration

3. Who can participate?

Anyone who meets all of these:

• At least 18 years old at registration
• At least 2 years of professional experience, with real expertise in any field (software, engineering, finance, legal, healthcare, science, design and more)
• Able to code, or skilled at building with AI coding tools like Codex or Claude
• Not a resident of Afghanistan, Belarus, Cuba, Democratic Republic of the Congo, Iran, Iraq, Libya, Myanmar, North Korea, Russia, Somalia, South Sudan, Sudan, Syria, Ukraine or Yemen

4. Can I participate as a team?

No. This is an individual event with individual prizes. Register once and submit one entry.

5. I already work with micro1. Can I join?

Yes. micro1 experts and contractors who meet the eligibility criteria are welcome. Only micro1 core team members, event administrators, judges and anyone involved in creating or testing the challenge are not eligible for prizes.

6. I registered but didn't get a confirmation email. What should I do?

Check your spam or promotions folder first. If it's not there, email hackathon@micro1.ai with the email address you registered with.

7. Do I need to do anything else after registering?

No. Once you've registered, wait for the event to start. You'll receive an invitation email with your challenge link at kickoff.

8. I missed the registration deadline. Can I still join?

Yes. Registration on the event page closes when the challenge opens. From then until submissions close, a direct challenge link is posted in the Event Details section of the event page. Use it to join and complete a short onboarding form.

9. What is the onboarding form?

When you start the challenge, you'll fill in a short form with your profile links (such as LinkedIn, GitHub or HackerRank) and some background about your experience. It helps us understand who is building what.

Dates and Schedule

10. When will the problem statement and instructions be available?

At kickoff. The full instructions, submission format, rubric and reference materials are posted in the Instructions tab of the challenge on HackerRank when the event starts. Before then, the event page covers the theme, rules and schedule.

11. Is there a kickoff call, recording or presentation?

There is no live kickoff call. At kickoff, we share teaching material on what makes a great eval, along with the instructions and reference examples. Use the written instructions as your main reference.

12. Can I start late?

Yes. You can start any time while the challenge is open, as long as you submit before the deadline. We recommend starting early so you have time to test and polish.

13. How much time should I plan for?

Plan to work across the two weekends. Keep your scope tight: a focused, well tested eval beats a large, unfinished one.

Building Your Eval

14. Do I solve an assigned problem, or choose my own?

You choose. Pick a real task in a domain you know well where you believe frontier AI models fall short. The instructions include examples, but you decide what to build.

15. Can I build an eval in any domain?

Yes. Software, finance, legal, healthcare, science, engineering, design or anything else you know deeply. Your expertise is what makes the eval valuable.

16. Do I need prior experience building evals?

No. We share optional teaching material and reference examples for anyone who wants them. If you've built evals before, you can skip them and start right away.

17. Do I need to know how to code?

Some coding ability is needed, because an eval is built as code. Strong skills with AI coding tools like Codex or Claude also work, as long as you can turn your expertise into working tasks and a grader.

18. Can I use AI tools to help build my eval?

Yes. You're welcome to use AI coding tools and assistants. You are responsible for the final quality, originality and correctness of your submission.

19. Will you provide starter tools or templates?

Yes. At kickoff we share reference materials to help you go from idea to validated eval in the expected format. They're optional but recommended, since they help you match the submission format.

20. Do I have to use a specific framework or format?

Follow the submission format described in the instructions released at kickoff. The reference materials show you exactly how to match it.

21. Which programming languages can I use?

We don't restrict languages or runtimes. Just make sure your documentation explains clearly how to run and reproduce your eval.

22. Which AI models should I test my eval on?

You can build and test on free or low cost models, such as the free models available on OpenRouter. To make your submission stand out, show how frontier models perform on your eval. Clear failures on frontier models are what impress the judges most.

23. Do you provide API keys or model credits?

No. Any model usage is at your own cost.

24. Can I use external APIs and my own paid accounts?

Yes, as long as we can reproduce your eval during judging. Never include API keys, passwords or other credentials in your submission.

25. How many tasks should my eval include?

We recommend 5 to 10 tasks. There's no maximum, but focus on quality: each task should be clear, gradable and meaningful.

26. How do I show that models fail?

There's no fixed pass rate you need to hit. Run your eval on one or more models, include the results, and explain where and why the models fall short. We expect you to impress us with meaningful failures on frontier models, showing a real limitation rather than a trick question.

27. What data can I use?

Use only information you're allowed to share. Public or synthetic data is easiest. Never include confidential employer or client data, private personal information or anything you don't have rights to use.

28. Can I submit something I built before the hackathon?

Your submission must be your own original work for this event. If you build on earlier work, clearly state what existed before the hackathon and what you built during it.

29. What if someone else builds a similar eval?

That's fine. If submissions are similar, the one with the clearest, most rigorous task design and grading scores higher.

Submitting

30. What do I need to submit?

Three things, all through the event page:

• Your eval as a ZIP file: code only, max 50 MB, in the format described in the instructions, including a README that explains what your eval tests, who it's for, why it matters and how to run it
• A video link: a walkthrough of up to 5 minutes, hosted on YouTube, Google Drive, Loom, Vimeo or similar
• A short description: a brief overview of what you built and what it shows about model limitations

31. What is the maximum file size?

50 MB for the ZIP file. It should contain code and small supporting files only. Don't put videos inside the ZIP. If your eval needs large files, host them elsewhere and include an accessible link and instructions in your README.

32. What do I need to make my eval reproducible?

Judges must be able to run your eval in a clean environment and check your main results. Include the commands, dependencies, data (or links to it), versions, expected results, and approximate runtime and cost.

33. Where can I host my video?

YouTube, Google Drive, Loom, Vimeo or any similar service. Make sure anyone with the link can open it without requesting permission.

34. How long should my video be, and what should it cover?

Five minutes or less. Explain the problem your eval targets, how it works, how it's graded and what it reveals about model limitations.

35. Do I need to show my face or use my own voice?

No. You can present without showing your face, and an AI generated voice is fine, as long as the explanation is clear.

36. Does my GitHub repository need to be public?

No. You submit your eval as a ZIP, so the repository can stay private. Make sure any extra links you include are accessible.

37. Can I update my submission after submitting?

Yes, as long as the deadline hasn't passed. Only your latest complete submission is evaluated.

38. Can I submit more than one eval?

No. Submit one eval. If you resubmit, only the latest complete version counts.

39. What should I check before submitting?

• Your ZIP opens and is under 50 MB
• Your README explains how to run and reproduce your eval
• No API keys, passwords or private data are included
• Your video link opens without permission and plays with audio
• Your description is filled in

Judging and Results

40. How are submissions judged?

Every submission is scored out of 100 by micro1's expert team using the rubric published at kickoff. A submission is scored only after it passes eligibility, completeness, integrity and reproducibility checks. Finalists are reviewed by micro1 leadership.

41. What makes a strong eval?

• It targets a real limitation that matters in practice, not a trick or gotcha
• Tasks are clear and unambiguous, with a defensible correct answer or grading standard
• The grader is reliable and doesn't produce false passes or false fails
• Frontier models fail for a meaningful reason that you explain
• It's original and reproducible

42. When will results be announced?

Winners will be announced on November 16, 2026, on the event page. Winners are also notified by email.

43. Will I get feedback on my submission?

Because of the number of submissions, we can't provide individual feedback.

Prizes and Opportunities

44. What are the prizes?

• 1st place: USD 5,000
• 2nd place: USD 3,000
• 3rd place: USD 2,000

45. How are prizes paid?

Winners must be able to receive payment through micro1's approved payout method in their country. Winners are responsible for any taxes that apply to them.

46. Does participating or winning guarantee a job?

No. Top performers will be considered for paid expert opportunities with micro1, at micro1's discretion and subject to additional review.

IP, Data and Privacy

47. Who owns my eval?

You keep ownership of your work. By accepting the micro1 Participant Agreement, you grant micro1 a licence to use your submission.

48. Can I publish my eval myself?

Yes. You're free to publish your own eval, for example on GitHub, after you submit.

49. Where can I read the Participant Agreement?

You can read it here: micro1 Participant Agreement. You accept it when you register, and it governs your participation and submission.

Rules and Conduct

50. What counts as cheating?

Submitting someone else's work as your own, copying existing benchmarks without disclosure, fabricating results, creating multiple accounts to submit more than once, or anything else that breaks the rules. Any form of cheating leads to disqualification.

51. Who makes the final decisions?

micro1 reserves the right to interpret the rules, determine final rankings and make final decisions on eligibility and disqualification.

Support

52. Where can I ask questions?

Email the micro1 hackathon team at hackathon@micro1.ai. Official clarifications are shared with all participants on the event page.

53. Who do I contact for technical issues?

For questions about the event, rules or submission format, email hackathon@micro1.ai. For login or platform problems on HackerRank, include screenshots and your registered email so we can help or route it quickly.

54. Is there a Discord or community channel?

Yes. A Discord community for participants will be linked on the event page. You can always email hackathon@micro1.ai too.

‍