Hackerearth Test creation automation — a UX case study

Many of our valued customers have been diverging from the use of prebuilt assessment templates, opting instead to craft customized assessments tailored to their specific needs. While this approach offers flexibility and adaptability, it comes at the cost of consuming significant amounts of time and resources. This departure from prebuilt templates also means that our product’s full potential may not be fully harnessed, potentially depriving users of the true value it can provide.

Furthermore, as users increasingly create custom assessments, we’ve noticed an undesirable consequence on the server end: a noticeable increase in load times. This decline in server performance not only hampers the efficiency of the assessment creation process but also negatively impacts the overall user experience. In essence, it results in a less-than-optimal assessment creation experience, undermining the quality and speed of interaction with our product.

---

Our current approach to presenting assessment creation options has inadvertently introduced a substantial cognitive burden on our customers. We offer a multitude of choices, including creating assessments based on skills, job descriptions, job roles, custom criteria, and utilizing pre-listed tests. While this extensive range of options is intended to cater to various user preferences and requirements, it has raised concerns regarding user comprehension, particularly among new users.

The complexity inherent in understanding the nuances of each assessment creation method can be overwhelming for individuals who are just beginning to explore our platform. This cognitive load, resulting from the need to decipher the merits and intricacies of each option, can pose a significant hurdle for users seeking a seamless and intuitive experience.

What has emerged as a notable consequence of this situation is that a substantial 70% of our customer base has resorted to using the “custom” option. While customization can offer a tailored approach, it also suggests that a significant portion of our user base may not fully grasp or be aware of the more automated and efficient ways to create assessments. This presents an opportunity for us to enhance user education and streamline the process of utilizing these automated assessment creation methods, ultimately enabling our customers to harness the full potential of our platform with ease.

During my discussions with our Customer Success Managers (CSMs) and members of the product team, it became apparent that there is a pressing need to consolidate the assessment creation process. This consensus has arisen from the feedback and requests made by a significant portion of our customer base.

Specifically, customers have voiced their desire to streamline the process by merging the currently separate workflows for skills, roles, and job descriptions into a single, unified flow. This request has surfaced repeatedly among our user community, indicating a strong demand for a more integrated and efficient approach to assessment creation.

To gain deeper insights into this issue, I conducted interviews with several customers. These conversations reinforced the challenges associated with our current setup. Customers highlighted that the proliferation of multiple distinct flows has made onboarding new team members a daunting task. It not only consumes a considerable amount of time but also contributes to a sense of complexity and confusion.

Furthermore, some customers expressed their perception that our user interface (UI) appears outdated. This perception adds another layer of concern, as an outdated UI can deter users from fully engaging with our platform and can affect the overall user experience negatively.

In light of these findings, it is clear that there is a compelling case for revisiting our approach to assessment creation. By consolidating the skills, role, and job description workflows and addressing the perceived UI issues, we can not only enhance the onboarding experience for new members but also improve the overall usability and attractiveness of our product to both current and prospective customers.

I conducted a comprehensive benchmarking analysis of the test creation processes employed by our competitors. In the course of this evaluation, a notable trend emerged: the majority of our competitors have amalgamated their test creation capabilities into a more consolidated and user-friendly journey. Specifically, they have streamlined the process by incorporating automated test creation options that revolve around job-related criteria, skills, and roles.

One common feature observed among our competitors is the provision to create assessments based on predefined job roles, specific skills, or other relevant criteria. These automated pathways simplify the assessment creation process by guiding users through a series of intuitive steps that are closely aligned with the typical needs of their organizations. This approach not only expedites the creation of assessments but also ensures that they are tailored to the specific requirements of the job or skill set in question.

Following in-depth discussions with our engineering team, I embarked on a collaborative effort to enhance the user experience and efficiency of our automated test creation process. One of the pivotal outcomes of these discussions was the development of a well-defined visual hierarchy for mapping questions, complete with a comprehensive array of parameters.

This visual hierarchy serves as a pivotal tool, providing a clear and structured framework for organizing questions and related parameters within the automated test creation process. By establishing this hierarchical structure, I gained a deeper understanding of the intricate interplay between various elements and factors that influence the creation of assessments.

Moreover, this structured approach facilitated the breakdown of the automated test creation process into distinct and manageable steps. Each step corresponds to a specific aspect of the question mapping, ensuring that the process flows logically and efficiently. This step-by-step breakdown not only simplifies the overall process for users but also allows for a more seamless alignment with the backend systems and mechanisms that power our platform.

In essence, the creation of this visual hierarchy and the subsequent division of the process into well-defined steps represent a concerted effort to optimize the functionality of our automated test creation feature. This strategic approach enhances both the user experience and the backend operations, ensuring that our platform operates with precision and efficiency while offering users a more intuitive and streamlined journey.

I have integrated all the acquired knowledge into the wireframe design and developed an interactive prototype. This prototype serves as a tool to collect feedback from both product managers and Customer Success Managers (CSMs), followed by collaboration with engineers for further refinement. This iterative process helps ensure the design aligns with product goals and user needs.

I have proceeded to develop the visual design based on the valuable feedback received. This design underwent user testing with key partners, during which it garnered positive usability ratings. In essence, the visual design was refined and validated through user testing with our key stakeholders, confirming its effectiveness and user-friendliness.

I provided the frontend developer with a comprehensive list of all possible states that the application or feature could encounter. Additionally, I ensured that they had a clear understanding of the specifications, which included details about how the application should behave or appear in each of these states. This thorough communication and documentation are essential to ensure that the frontend developer can accurately implement the desired functionality and user interface.

I shared all the invaluable feedback with the frontend developers to refine the design down to the pixel level, ensuring that every visual element and detail aligns precisely with the intended design specifications. Additionally, we rigorously tested the functionality in a staging environment to guarantee that not only does the design look flawless, but it also functions seamlessly in a real-world setting before deploying it to the production environment. This meticulous attention to both form and function is crucial for delivering a high-quality and polished user experience

After a few months of implementation, we began to observe a significant increase in the utilization of automated tests within our product. This shift indicated that our customers had started to grasp the true value proposition of our offering. In other words, they had come to recognize and appreciate the benefits and advantages of incorporating automated tests into their workflows. This positive trend underscored the growing acceptance and adoption of our product within our customer base