City of Chicago.org: Improvement in information architecture
The official City of Chicago website (Cityofchicago.org) is an information rich website used by business owners, city residents, and visitors alike. View Full Report
My Contributions: Usability tests, Sketches, Low and Mid-fi prototypes
challenge
The way the information is currently presented is convoluted and at times overwhelming. With the overall goal of improving the navigation of the City of Chicago website, we had three subgoals:
improve the findability of topics on the primary navigation menu
increase satisfaction with the organization of the CityofChicago.org website
determine the effectiveness of our prototype information architecture
Methods and Results
initial Usability Test for current website
Tasks/task protocol:
We chose two main tasks for our usability evaluations:
Paying a parking ticket
Finding information about bedbugs
After developing an interview script and usability evaluation protocol we conducted an initial usability test using the two tasks to provide a baseline measure of usability, checking for sources of error, and gauging satisfaction.
Results
All five participants were able to finish the parking ticket task quickly (four of five in less than 30 seconds, with an average time of 27.4 seconds) and none of the five were able to find the bed bug information within the five minute time limit. However, two of the participants actually followed the right path to the bed bug information by navigating to the Public Health Department page, but failed to notice the bed bug icon present there and thus continued onward to more pages. One of those participants commented that “bed bugs seem like a public health issue." The participants grew increasingly frustrated as they navigated through the menus, stating “this site is terrible." One participant even started to question, “is this even possible?"
An error was defined to be the participant taking the wrong path. Incorrect clicks that occurred within already incorrect pages were not counted, including clicks back to the homepage. Participants did not make any errors in navigating to the parking ticket page. There were many errors for the bed bug task with the number ranging from 6 to 12 with a median of 7.
All participants thought the parking ticket task was “very easy” and the bed bug task was “very hard." The responses to the question about understanding of the site were mixed. Participants were satisfied with the navigation to the parking ticket page but not to the bed bug information.
Open Card sort
We recruited 11 students, colleagues, and friends to participate in an open card sort using OptimalSort. In preparation for the open card sort, we conducted a partial content inventory of the webpages involved in the site’s overall composition, focusing in particular on those pertaining to our test tasks. This process yielded over 150 cards; to reduce the potential for participant fatigue we reduced the number of cards to 104 by eliminating redundant pages and removing cards that would have been too difficult for participants to understand without context.
Each participant was asked to sort the cards into sensible groups and then name their groups. The card sort was unmoderated and completed remotely on the participant’s device . We analyzed the results using OptimalSort’s Best Merge Method dendrogram and Similarity Matrix and worked as a team to optimize the number and content of the user proposed groupings in a shared spreadsheet (“Interpret your card sorting,” 2017).
Results
The completion times ranged from 7.5 minutes to 1 hour and 27 minutes; the median time was 21 minutes. Several participants commented that the number of cards was “overwhelming.” The number of groups created by participants ranged widely from 8 to 20 with an average of 12. This large number of groups from the card sort did not lead to a clear-cut menu organization. The lack of a most popular grouping meant that the results needed vigorous review. This required us to use OptimalSort’s tools and our expertise to judge where some ambiguous cards should be placed.
The Similarity Matrix was examined to understand how participants grouped various pages; this showed that the strongest associations for bed bugs were with “Public Health” related topics. These were the only cards that bed bugs were paired with by more than half of participants. The Best Merge method dendrogram suggested that this grouping should be labelled “Health” or “Health and Wellness". The next topics most commonly grouped with bed bugs were home repair and mattresses, followed by condo ordinances, renting, and safety.
The parking ticket page was most often grouped with other transportation topics. Interestingly, it was paired in the same category with “Pay Water Bill” by 4 out of 11 participants, which pointed to the possible utility of having an additional navigation menu organized by task. The dendrogram revealed that participants named these groups “Transportation” and “Parking & Transportation.”
Lo-fi prototype
Some of the suggestions that we received as part of the follow-up questions were used in the lo-fi design. Specifically, these were that the site needed “better phrases or target words” and that it should “list bed bugs under different categories." These comments affirmed the need for new menu labels and the reorganization of content.
The new organization of the main navigation menu is based on a specific group of users: regular citizens. The official city website reflects a structure based on government names and hierarchy. Though important to consider, we felt our approach (derived from the card sort results) would be more understandable and therefore more learnable for the typical city resident or visitor. The result was a more consolidated grouping of pages, with six main categories and each of those containing two to four subcategories. The new categories we created are shown in our mid-fi prototype main navigation menu.
Prototype creation and Evaluation
After further combining groups of topics and determining a new information architecture from the open card sort, we created a low fidelity wireframe using the software Axure. We focused on building the main navigation menu and organizing the topics into logical submenus. View Prototype.
To create the top navigation menu and the groupings within them, we followed the large menu grouping guidelines outlined by the Nielsen Norman Group (Nielsen, 2009):
Chunk options into related sets, keep a medium level of granularity.
Use concise, yet descriptive labels for each group.
Differentiate labels.
Order the groups.
Show each choice only once.
The lo-fi prototype allowed us to quickly test the new navigation for the two tasks. We used a similar evaluation protocol to our initial usability test for consistency. However, to validate the new architecture we also asked the participants to find some additional, commonly accessed pages. These tasks were:
looking up building permits
reporting a pothole
applying for a job
getting a business license
Results
Three participants reviewed the lo-fi prototype and all were able to complete the two tasks (parking ticket and bed bug) quickly and without errors. Additionally, all participants indicated in the post-task questions that the navigation was understandable.
For two of the three participants there was a slight delay in determining that the parking ticket task would be under the main navigation menu category “Transportation.” However, all of the participants were able to determine that given the menu alternatives, “Transportation” was the most sensible location for the page. The average completion time for this task was 21.3 seconds.
Completion of the bed bug task was markedly easier for participants while using our prototype. The average time on task for finding the bed bug information was 15.6 seconds. It was obvious to the participants that this topic was located in the “Health & Wellness” menu.
There was general satisfaction with the prototype layout and navigation to the target pages. Participants were also able to locate the pages for finding out about building permits, reporting a pothole, applying for a job, and getting a business license. One participant commented that he was able to deduce where the desired page would be found because he could “compare the menu categories easily.” These results suggested that our design was an improvement from the original city site.
Mid-fi Prototype
Implications for design
One of the participants was also part of the initial city website evaluation; during that test he had used the side navigation (titled “I Want To ...”) to quickly find the page to pay the parking ticket. The lo-fi prototype did not have this sidebar menu, so he was initially slowed by looking for it. The participant did complete the task successfully by using the main navigation menu, but mentioned that he thought it was a lot faster using the side navigation menu. Based on this feedback, that side navigation menu was added to the mid-fi prototype. View Prototype.
Usability Testing
All five participants were able to complete both tasks successfully. The order of the tasks had no bearing on the results.
Four participants completed the parking ticket task in less than 30 seconds and one completed it in less than a minute. The person who took the longest had followed the wrong path initially. This was the only error for this task. He looked for “Police,” reasoning “that’s who writes the tickets." When that path did not end with the parking ticket page, he turned his attention to the left side of the page, where he saw the “Parking Ticket” choice in the side navigation menu and followed that path successfully.
Two of the participants completed the bedbug task in 20 seconds or less, the other three took between one minute and one and a half minutes. The two quick results were because the participants looked in the correct menu (“Health & Wellness”) first. They chose that menu item because “bedbugs are a health problem." The other three looked in different menus (“Community” or “Building Management”) because their first instincts were to think of bed bugs as a housing or social services problem. After not finding any links to bed bug information in the “Community” or “Building Management” pages, these participants returned to the home page and followed the path through the “Health & Wellness” menu.
Though the users were successful with both tasks their satisfaction level with finding the pages was not as high as expected. Two people had a “Neutral” response regarding how easy or hard it was to complete the parking ticket task. The other three thought it was “Easy” or “Very Easy." The bed bug task had one respondent that rated it as “Hard”.
Four of the five thought they had a good understanding of the site and reported overall satisfaction (“Satisfied” and “Very Satisfied”). The prototype’s limited functionality and detail on the “wrong path” pages may have contributed to the less than fully satisfied responses. Participants expressed desire to see a more fully functional prototype, where all of the links are active and the menus behave like a finished website.
Conclusions
We saw an improvement over the results of the initial usability evaluation conducted on the original website, where no participants were able to complete the bed bug task. Thus, our results suggest it is possible to take topics that were very difficult to access and improve their findability without significantly decreasing the accessibility of commonly visited pages. However, not all the results were entirely positive. Even though the parking ticket page remained findable in our new organization, participants did think it was harder to complete the task. In the future we would further explore the tradeoffs involved in balancing the findability of popular and less popular content.
Initially we considered whether the new design should include the secondary navigation menu present on the current version of the city website. Our lo-fi prototype did not contain a side navigation menu, and user feedback indicated that this was indeed a useful part of the original site that needed to be reincorporated. After adding it to the next iteration, we observed that the secondary navigation menu (organized by task) improved the usability of our mid-fi prototype design. Therefore, this large, complex site benefitted from design patterns such as quick links that provide easy access to many topics without overwhelming the user.
Our card sort was less instructive than we had anticipated. Due to the smaller sample size and participant fatigue from working through such a large card sort, the results did not always indicate strong or exclusive group relationships between the cards. Group card sorting may have been a more effective approach for this project. The unmoderated testing environment meant that we missed out on participant comments that could have given their groupings more context. Understanding why participants thought certain content belonged together could have ameliorated the process of translating the results into a new architecture (Nielsen, 2004).
Qualitative feedback on prototype and suggestions for improvement
All of the participants generally liked the design, and one commented positively on its “open” feeling. In future iterations of our prototype we would include additional (contextually appropriate) paths to important pages. For example, we would create a path to bed bug information through the “Building Management” menu because several participants thought of bed bugs as a housing issue. Feedback from participants also suggests that we should increase the visibility of the side navigation menu, which we could achieve by making this menu a separate color from the rest of the page.
Considerations for future work
There are a number of additional tasks we would do if we were to continue developing a new information architecture of City of Chicago website. First, speaking to internal and external stakeholders in order to collect more detailed information on the site would help us get a better understanding of the content of the site as well as the traffic flow. We would also perform a site-wide content audit in order to remove redundant, out-of-date, and trivial content that is currently impairing findability (Boag, 2015). Furthermore, conducting a competitive analysis would also help us understand what kind of presentation of content is being used across other sites.
With such an immense website, there probably is not one best way to organize the information architecture. We would want to create and test a secondary navigation design, specifically organizing the content by user groups.
There would also be benefits in doing usability tests with a broader audience, as well as include more tasks. This could increase the confidence with generalizing the task results to the rest of the information architecture. We would also iterate on the mid-fi prototype, developing it into a hi-fi version for additional testing.