It’s been a little over a week since ManoMano’s first ever Hacking Weeks have concluded and now that I’ve had some time to reflect on how things went, I’d like to take you through this adventure, its challenges, learnings and promising next steps.
Introduction
You have probably heard about this concept but it wouldn’t hurt to just take some time to shed some more details on the what and the why of the Hacking Weeks.
To begin with, innovation or hacking or brainstorming days or weeks, call them however you like, are essentially giving the teams that are normally churning out new and exciting features for their customers, the time they would need in order to try out those crazy or out of the box ideas that might keep nibbling at the back of their mind.
What if we did that? What if we tried this? What if we had some time to do a PoC for this seemingly cool and innovative tool that would bring us a lot of value? What if we took some time to try out this new process? What if we had the freedom to try out anything that crossed our mind and try and make a success out of it?
These are some of the questions that the Hacking Weeks try to meet and give room for in order to find their answers. For a limited amount of time you take your mind off of the “mundane” work and put all your efforts and creativity into exploring whatever crosses your mind that could improve the life not only of your team or company but also of the customer. It’s a win-win situation (the best kind!).
It is with this idea in mind that we embarked on our first ever Hacking Week inside ManoMano. As a member of the QA team (denominated as Qraft inside the company) I had high hopes for what was to come especially given that there’s always a plethora of ideas going on in our heads related to continuous improvement and trying out the most novel of them is surely something that we have an itch for.
One of the beauties of this initiative is that you get to propose the topics you wish to work on. Surprisingly, you’ll often find a passionate team rallying around your idea, ready to dive headfirst into the challenge. What’s not to love about that?
Choosing the right topic
Working a lot in our day to day interactions and collaboration with the teams on back-end testing, focusing especially on APIs and their interactions, the search for our topic wasn’t a strenuous one. We had heard just a few months back that Postman is working on implementing an AI dubbed Postbot into their tool. That was the sign that we needed. A lot of questions crossed our mind in relation to this new AI powered tool that is native to Postman so naturally we wanted to explore its possibilities, try as much as possible to find our answers, become aware of which of its use cases might prove valuable to our teams, see what limitations we find on the way and hopefully reach a conclusion that would empower us to push towards its adoption on a company level.
Together with my colleagues Anne-Laure Gaillard and Anurag Pandey, we embarked on the Hacking Weeks having this clear goal in mind and a bunch of questions: is Postman’s new AI assistant going to help us out and improve our lives as QAs or would it be better to revisit it in some time? Could we use it for facilitating the automation of the API testing? Could we use its capabilities to maybe cover ground that was previously left uncovered? How could we use this tool for API exploratory testing?
Photo by Ethan Sykes on Unsplash
Hands-on exploration time
Our preferred way of working was autonomously and asynchronously. We still agreed on having a daily touch point of variable duration in which, just like an Agile team, we’d talk about our progress on the different topics that we split the endeavour in, see if we have any roadblocks or if any of us might need some help registering progress with their topic.
Even though it is still in its Open Beta phase (at the time of writing this article), it became apparent pretty quickly that Postbot was already a working part of Postman (the harder thing was finding it -> Tip: it’s the small purple icon that appears in the top right hand corner when you navigate to the Tests tab of your request)
Out of the box, Postbot is giving you some nudges towards its capabilities such as: adding tests for a response, visualizing the response, saving a field from a response or just fixing the tests you might already have. The appeal of this is also that, similar to other AI assistants, it uses natural language, which makes it easier to interact with it.
We are also excitedly looking forward to future capabilities that Postman will bring to its AI assistance, things such as: writing documentation for API calls, build a test suite from scratch through the Collection Runner, summarize test data reports to help you focus on the most critical aspects, debug API calls on your behalf, based on your intended outcomes or help you search for the right API calls in the context of the workflow you are trying to achieve.
Postbot AI feature is particularly beneficial for exploratory testing sessions focused on APIs. We can dynamically visualize API responses, gaining valuable insights and facilitating a deeper understanding of the system’s behaviour — Anurag Pandey
Thus, after creating a separate workspace and importing some collections that we had prepared for this exercise, Postbot was starting to hint at its value.
Its speed and ease of use is indeed on par with other more famous AI assistants such as ChatGPT, which I think decidedly marks a before and an after. Of course, having it embedded in Postman and ready to have its outputs ready to go is definitely a quality of life improvement that we as QA appreciate.
After taking some time to see exactly the way in which the most basic use cases of Postbot could help, we started going further and thinking about its use in our API-focused exploratory testing sessions.
Before, what we used to do was organize sessions similar to guided testing ones in which we selected a few endpoints that we wanted to perform exploratory testing on. Using the Pomodoro technique, we’d each in our own way try out different techniques (such as fuzzing or applying security-focused testing templates among others) to uncover defects and irregularities.
Digging deeper
As we delved into Postbot’s offerings, we managed to use its capabilities to help us visualize the responses that we get (and with that export them to our stakeholders which would give them the much needed visibility over the ROI) but also come up with new tests that we might have missed when exploring the different selected endpoints.
Photo by John Schnobrich on Unsplash
Day after day we progressed on our topics of the Postbot integration and also our API-focused exploratory testing sessions. This soon involved other teams such as Security or IT Operations, as we needed external help to validate some tests that Postbot was coming up with and assess their applicability on the ManoMano infrastructure.
As the Hacking Weeks drew to a close, Postbot had become an integral part of our workflow. We acknowledged the need to review its outputs rather than blindly copy-pasting, but overall, the integration proved seamless and efficient.
The end of the Hacking Weeks was marked by a demo of the things that we have accomplished during the time we were given to experiment with our chosen topic. In the end, all of it was done to serve more than one purpose and what we are eagerly looking forward to is not only the applicability inside of a day to day scenario of our findings with Postbot but rather to the whole plethora of outputs from all of the teams across the company and how these will feed into the innovation backlog in the short, mid and long run.
Reflecting on the experience
Having had the time and room in order to run this PoC on Postbot was definitely a breath of fresh air that we wouldn’t have had the time to take between all of the things that we are handling each day. We are not finished with it, on the contrary, it’s only the beginning for what we already believe might be a valuable improvement to our API testing techniques and the time that we spend on covering the necessary bases with each team.
Peering outside of our topic, we were really looking forward to seeing what other colleagues managed to achieve at the end of the Hacking Weeks and the truth is that taking this time every now and again to explore new ideas is paying through and through.
The best moment of this initiative? Without a doubt for me, it was to act like a security hacker during Hacking Weeks. It was almost ironic but so cool. — Anne-Laure Gaillard
In his capstone book Drive: The surprising truth about what motivates us, Daniel Pink talks about Atlassian (you know, the people behind JIRA and Confluence among other things) and how they, once a quarter, on a Thursday afternoon, tell their developers: “For the next 24 hours you can work on what you want, with whoever you want, all that we ask is that you show the company the results of your work in a fun meeting”. Daniel Pink continues by saying that in just one day each and every quarter, Atlassian managed to bring forward through this exercise a whole array of ideas for new products and fixes for old issues that otherwise would not have seen the light of day.
To conclude, we already are looking forward towards the next iteration of the Hacking Weeks and it’s honestly already challenging picking out a new idea as they all seem exciting.
We ❤️ learning and sharing
If you’d like to get in touch or just talk about QA in general, I’m always reachable through my LinkedIn profile. Drop me a line! Whether you had a similar or totally different experience, I’d love to hear about it.
Exploring Innovation: ManoMano’s Inaugural Hacking Weeks was originally published in ManoMano Tech team on Medium, where people are continuing the conversation by highlighting and responding to this story.
