Understand

Dig into Prior User Experience Data

Preparatory Research We conducted some preparatory research before we began work to ensure that we understood the stakeholder requirements and any existing pain points they’ve experienced in the past when planning for previous years. In this preparatory research, we reviewed previous years’ roadmaps to understand the typical size of the initiatives that have been sought/approved and how to propose them in a way understandable by CMS leadership, reviewed the team’s meeting notes in Confluence and any parking lot items we’d identified during other initiatives throughout the year, and read through new policy details about Medicare that we would need to account for in the upcoming year. From our preparatory research, we noticed that prior initiatives were only shared with estimated effort and a “how might we” statement to guide our work. We also identified two key policy changes that directly impacted Medicare and the content we’d need to display on the website: (1) a life experiences project from the Office of Management & Budget to improve the customer experience around people approaching retirement and (2) a change as part of the Inflation Reduction Act of 2022 in payment structure for prescription drugs that allows costs to be split out through the entire year instead of upfront. Stakeholder Discussions Alongside our preparatory research, we facilitated discussions with various stakeholders to see whether there were other initiatives we can support, hypotheses about the user experience we could gather data to assess the validity of, and any ideas that had already been discussed as a potential plan for 2024. Many of these discussions were facilitated during PI Planning sessions as we shared our plan to conduct discovery and begin roadmapping for the next program increment out with the other partner teams involved. In these discussions, we also invited them to share ideas/hypotheses they had and to join in on ideation for particular initiatives that impact their own work.

Define

What does the Medicare.gov Website Need

Creating a Plan Find all available data sources Note: Create a research repository. This work Gather questions and hypotheses from the data OE v. Rest of year Discovery Discovery Sources Adobe Analytics data Qualtrics survey data QuantumMetric heatmapping Python NLTK for sentiment analysis, Prior research data (primarily around user interviews) Findings Homepage lower satisfaction recently Reporting Out Since we operated in sprints, I put together a biweekly report of the key findings from that sprint. Update confluence at the end of each sprint. Documentation Show your work: spreadsheets of raw data Find insights: list out the coolest stuff I saw and start a conversation with that. Use this as a time for blue sky ideation. Find all available data sources Note: Create a research repository. This work Gather questions and hypotheses from the data OE v. rest of year One sprint per deliverable, but alongside other work.

Design

Creating a Separate UX Backlog

From our discovery, we recognized that there needed to be a shift from larger initiatives to more iterative improvements across the site. In order to move to a continuous improvements framework, we needed to collect all of our ideas into one place to help us keep track of them all. However, previous attempts to store our UX backlog in JIRA alongside the Engineering backlog were rejected, as JIRA was only used for items already approved for work. While this approach was not ideal and introduced more places for disconnects between project teams, we decided to maintain our UX backlog in a separate location using an Airtable base. This allowed us greater control over the data structures we used for prioritizing these ideas and allowed the entire team to have a space where they could propose new ideas for UX initiatives. To make sure there was still stakeholder visibility of our UX work, we kept the backlog accessible to all teams and used our existing communication channels and facilitated discussions during sprint planning to run through the top 5 ideas on a bi-weekly basis to create conversation around those ideas, and seek approval to add them to JIRA to track our team velocity accurately. In order to create this backlog, we started with a form to collect new ideas and used it to populate our backlog with the existing ideas already documented.

Develop

Building Out the Backlog/Roadmap

Using a Now, Next, Later approach to avoid keeping us held down by the timeframes and allow us to work ahead. As we further get into the new year, we can move toward using a Gantt chart to demonstrate the roadmap more formally along a timeline to best estimate potential launch dates. The prioritization framework is using the RICE method to calculate overall score based on the reach, impact, confidence, and effort to finalize these updates.

Deliver

Socializing the Roadmap for Stakeholder Approval

Once we prioritized the newest UX activities using our framework and scoring model, we wrote documentation to show the raw data, key insights, and actionable recommendations for how we could improve the Medicare.gov site. This documentation was used to communicate all of the data that was collected and how it could be used for iterative improvements in the coming year. With the updated scoring calculator mostly filled out, we were able to identify the top ten most impactful ideas and share them with our stakeholder team for an initial read on CMS' confidence in the solution as well as any background knowledge on the proposed initiative to refine the roadmap. We updated the format of the data structure underlying the roadmap database to include all of the necessary fields for automatically importing the initiatives into JIRA as Epics. We also built in automations to save us time we'd have spent completing redundant tasks, like calculating scores for each of the prioritization facets based on user input and determining the overall prioritization score for the based on those facets and alignment with CMS strategic pillars. Once we'd completed merging our research repository into the same tool as the roadmap, we were able to update ideas within the roadmap document using recommendations that came out of specific research study reports to ensure no ideas were lost from discovery/research reports as well as being able to link the initiatives to the data that support them.