Recently our CEO came to me and said “Why don’t we do a hackathon on AI so the team can further immerse themselves into building with the latest AI tools and get even more familiar with the technology, tools and further iterate on our AI-first approach”. Unknown to him, he hit a sore point for me – I really dislike hackathons. So I proposed we do an AI focussed sprint instead. After some arm-twisting and discussion over what is wrong with (most) hackathons, he agreed that the AI sprint was a better approach and would lead to real deliverables backed by real requirements.

‍

Why I do not like hackathons

For me the difference is much more than just the name. My experience is that hackathons tend to have a few focused groups, while most other people end up just wandering around in a caffeinated or intoxicated state. This reduces the chances of the desired random get-togethers of people across the company to generate new ideas. In fact, most people are busy before the hackathon and show up with a vague idea that they may try – generally these efforts end up in early beers.

Furthermore, hackathons can reflect wider issues in the company:

• Lack of an innovation culture
  
  • Hackathons are regarded by many management teams as almost “playtime” or a fun competition for developers, with little expected in terms of product improvements or innovation. Such a token commitment to innovation is reflected in hackathons that are held over a single day. 
• Undervaluing professional software development
  
  • One day hackathons underestimate how hard it is to develop software and the work required to do it well – “they can just hack something together quickly” is an attitude that does not indicate or promote a good professional culture of innovation and building products.
  • Senior management may not attend the hackathon or do so briefly, unlike other company get-togethers. 
  • A hackathon generally includes beers and pizzas, whereas other company get-togethers such as Quarterly Business Reviews tend to have much fancier and more formal arrangements!

‍

What can be good about hackathons

There will always be some teams that meet beforehand and plan their work. They feel strongly about something they want to develop and use the time and visibility of the hackathon to develop a POC in the hope that it can get funded later. These tend to be the most productive outputs in a hackathon, but this type of commitment from those involved should be encouraged all the time and not just for a periodic (usually annual) hackathon. Such an effort is also a little closed, so may not achieve the goal of widening the collaboration across teams.

Other good aspects of a hackathon can include:

• Providing a chance for developers and the rest of the company (such as PMs, sales teams, UI designers) to collaborate and this is valuable in cases where they may not usually interact or interact only in very specific circumstances allowing them to:
  
  • Share problems and bounce ideas around (requires some up-front preparation on how best to enable this)
  • Get a concerted effort going (connect with each other and find shared purpose)
  • Learn a new technology with others in a collaborative environment (although the nature of hackathons means they can become a contest and less collaborative)
• A hackathon that is focused on a specific result may achieve good results, if there is sufficient preparation on the goals and arrangements (such as collecting feedback), e.g. a hackathon where customers and integrators are invited to try out a company’s new API prior to release can have many benefits in evaluating that API.
• A hackathon for students, e.g. sponsored by a company as part of their recruitment process with engineers to assist students, can give the company insights into prospective hires and boost their image. It gives students a chance to develop skills, solve problems under pressure and to benefit from collaborating with experienced engineers.

‍

What we did – an AI sprint, not an AI hackathon

Given the importance of getting our team even more immersed in AI, we considered it worth the disruption to ongoing roadmap/sprints  to take on a dedicated AI sprint. This investment was considered vital given the pace of change in AI technology and we defined an AI sprint dedicated  to accelerate our understanding of recent AI developments and to investigate what new capabilities might be added to our roadmap as a result.

It’s worth noting we’ve been building AI powered logging capabilities since our inception and have regularly performed core research such as LLM benchmarking to see how AI performs with analysing Log data. 

Here’s what we did:

• An Epic was created for the AI sprint, so everyone could see what was proposed, including the CEO
• Stories for the Epic were proposed ahead of the sprint by all members of the team
• The process was open to all ideas related to the AI topic 
• The stories were reviewed and prioritised by PM and development leads in a lightweight manner as per a normal sprint, but this time even the CEO gave input. Curating the list in this way avoided a free-for-all set of tasks, but also involved the team.
  
  • There were more stories proposed than could be completed, so the proposer was able to advocate for their story as part of the review.
  • People signed up for the stories.
  • The aim was to assign people to stories they proposed or were interested in.
  • The assignments were not done per the normal sprint teams to increase collaboration.
  • Stories were written with a view to create working software that could be included in the product – code was reviewed per the normal process and released to production directly or behind a feature flag in cases where we wanted customer feedback before going fully live. Note this depends on your agile process having tooling that allows selective release of features to customers and staging environments to test the features. It also relies on a  quick decision making process with the PM being the ultimate authority on what is provided to users.
  • Stand-ups were done as usual
  • A sprint demo was done as usual at the end of the sprint, but it had to be extended, as we had so much work that could be shown!

‍

What were the results?

In terms of the product, we got working code in the product for new features or the basis of new features. Not just hacky code shown at the end of the hackathon when people were tired, uninterested, intoxicated or full of coffee. Specifically, because of the curated approach to creating a list of stories, we created the set of features below that could be released to production or iterated on in future sprints:

• New AI features to reduce toil were added to the product. Examples include 
  
  • a new AI assisted Naming service using an LLM in AWS Bedrock with engineered prompts to suggest names and descriptions as part of configuring features in the product – this is outlined in an earlier blog post.  
  • a new AI Dashboard Builder to remove the toil in building dashboards, e.g. of having to write a query, as described in a recent blog post
• New log analysis and summarisation features ready to demo to customers for feedback. This was built on a new service that used our log engine to search logs quickly, supported by an LLM in AWS Bedrock where we designed prompts to summarise logs - these will appear in a new UI experience soon.
• A new AI assisted Investigation Report service which generates a report and delivers it to a user a few minutes after the creation of an alert. This feature uses our log engine in conjunction with an LLM in AWS bedrock to gather and analyse log data relevant to the alert and then suggest the appropriate actions to take.
• New Bronto MCP server which offers a suite of tools to extract insights from log data by helping users identify relevant datasets, query log events and compute meaningful metrics. Further details are provided in this blog post
• New areas identified for use of AI, with follow up Epics for how to incorporate them into the product
• New core functionality was identified that will enable us to exploit AI further in the product

In terms of the team, we got:

• A shared purpose across the team with lots of sharing of knowledge around tools to use and common issues faced such as configuring tools correctly, using the correct versions of libraries and approaches to writing prompts(and this is valuable at this stage of AI tooling). This happened much more widely across the team than I would expect from a hackathon.
• A baseline of new AI knowledge was created across the team and the sprint enabled the sharing of a significant amount of knowledge from those engineers with deep AI expertise.
• Specific areas where people are interested in developing expertise, such as writing better prompts, ML models, capabilities (and limits) of LLMs.
• Enthusiasm across the team, with lots of extra hours worked.
• Everybody from developers to the CEO are thinking about how we should and should not use AI from this point onwards. 

In broader terms, the AI sprint strengthened the company’s belief in our agile process and using it as a key part of a culture of innovation and valuing the professionalism of our team. In terms of inclusion and valuing professional software development, the AI sprint involved the engineers, but also the managers, customer support, PM and the CEO in reviewing stories, attending standups, helping on code and taking part in the sprint demo. And yes, there was beer and pizzas after the demo, but also a nice team dinner a couple of weeks later!

‍

Summary

I have shown how a sprint focussed solely on AI allowed us to get a number of valuable product outcomes, with a high degree of collaboration and enthusiasm across the company.

So why would you want to do a hackathon and not an AI sprint? If it is because you don’t think your development process is suitable, then perhaps you need to understand why that is the case, re-read the agile manifesto or the more cynical (and often accurate) interpretation and take action if it is unwieldy, bureaucratic or not supporting innovation. If you think your agile process is good and you still do not want to do this, then you should ask if it is related to the company/organisation culture, e.g. is your culture resistant to committing to this topic or to innovation in general.

Our experience is that if you make a (senior) management and engineering commitment to a dedicated sprint for all, then it will be taken seriously by everybody and they will put in a lot of extra time to learn and contribute, accelerating innovation and the use of AI within your product and company. Will we do it again – absolutely.

And a final word from our CEO “ This is so ******* exciting!! What we've achieved in such a short time in terms of new product capabilities & direction, tooling, knowledge transfer and a better understanding of the lay of the land is incredible. We are at a very exciting time and it is very cool to be able to apply this innovation with our domain expertise to drive real value for our customers. BTW I'll never mention the word 'hackathon' again.”