2015–2020

Lucidchart Visual Reasoning

This case study outlines my work as an individual contribute and a manager between the years of 2015 – 2020.

The Problem

As Lucidchart’s product strategy evolved to become more data-centric, Lucidchart needed a comprehensive toolset to allow customers to visualize, interpret, and take action on their data-generated diagrams.

My Role

At the onset of this effort, I was a senior designer working on a cross-functional team of engineers and product managers. Over time, I moved into a UX management position to oversee several other designers developing new solutions in this space. Throughout the effort I conducted research, built prototypes, delivered high-fidelity UI solutions, presented to leadership, and influenced business strategy.

The Diagramming Market, 2015

When I joined Lucid in 2015, Lucid was edging out other players like Gliffy and Draw.io to keep their eyes on the prize: the multi-billion-dollar Microsoft Visio. Each of these companies was battling for the acclaimed SEO prize of "best flowchart maker." All of our work centered around making boxes and arrows easier to drag out, align, and export.

Assumptions

At the time, we operated under the following assumptions (backed and validated by research over and over again):

  • Source of Knowledge: Because most tools didn't have connections to other platforms, customers consumed information and recreated it from their head. If it existed on paper or in a spreadsheet, they manually copied this information into a more legible diagram format.
  • Customer Role: As they performed the manual task of copying or regurgitating information, they played the role of a Creator. They were outlining, documenting, and creating information that likely had not been documented before.
  • Diagram size: A "large" diagram was considered to be a few hundred shapes, and could usually be understood and consumed by simply reading through the diagram.

For example, at the time this would have been considered "large" diagram:

https://d33wubrfki0l68.cloudfront.net/4532f852f7e1ee4e64e56c317cde73900e12b60b/1815c/lucidchart-visual-reasoning/untitled.png

As we'll see in future developments, this would turn out to be a fairly simple diagram.

Primary Goals

During these early years, I interviewed several dozen Lucidchart customers in a cross-functional group called the User Insights Group (UIG) to understand customers' industries, usage of our products, and priorities. We crafted several personas that were used for training within the Sales and Marketing departments. From these conversations, consistent goals and needs emerged:

  • Visual presentation and aesthetic quality of diagrams
  • Flexibility of expressing variations of ideas
  • Ease-of-use to connect shapes with lines and arrows
  • Speed of generating and capturing ideas and content

You can hear these assumptions being validated and these needs emerging in the customer's own words:

A lot of times what I will do when I'm creating a flow chart is I'll actually make a bolded list in word and ... I'll basically break it down as an outline kind of like I would do for a massive research paper when I was back in school. I'll say, "These are the high points I need to hit. I need to hit these in the middle and inside those middle points I need to make sure I capture these three steps of this process." Once I've done a little outline I then start going in with basic shapes. I usually don't do anything besides basic shapes. ... the fact that I don't need to waste any time in Lucidchart fighting with anything and I can very quickly create arrows, shapes, arrow shaped text—it's a very natural flow that I don't get anywhere else. — Lucidchart Customer, 2016

The source of knowledge is coming from his own head. The diagram size is modest. He's playing the role of the creator. What he needs to accomplish that is fairly straight forward: to get his ideas into a diagram as simply as possible, and flexible boxes and arrows the connect those ideas.

Given the state of the market and the customer demand, Lucidchart was meeting the need of quick, simple, visual thinking.

Diagrams from Data, 2017

Lucid began to innovate and experiment with other methods to create diagrams. What happened if the source of knowledge wasn't your own knowledge, but something else? Some internal engineers began experimenting with generating diagrams from files like a CSV or Microsoft Excel—or even a live API like Amazon Web Services (AWS). As one engineer connected to AWS to create compliance diagrams for their own need, Lucid recognized the potential and continued to drive it forward.

https://d33wubrfki0l68.cloudfront.net/821c61f6d209ed39399ae755048cd901cc3293b5/15f60/lucidchart-visual-reasoning/node-experience.gif

Assumptions

Using data (such as AWS) as a starting point for a diagram presented vastly different circumstances for generating and documenting information. For example:

  • Source of Knowledge: the information no longer lies in someones head, but may be able entirely foreign as it's imported from an external source.
  • Customer Role: As Lucidchart automatically imported data, or even generated entire diagrams, the customer was no longer the creator of the information but instead a consumer of external information.
  • Diagram size: All previous understanding of diagram size was soon dwarfed as customers could bring in potentially infinite amounts of data. In AWS, average infrastructures have a few thousand resources with tens of thousands of connections. At this point, the web browser is running out of memory needed to generate and display all the information.

Under these assumptions, a "large" diagram now starts to looks something more like this:

https://d33wubrfki0l68.cloudfront.net/945ef89ec7bae4c8c5018838a26013fee84917d6/2732a/lucidchart-visual-reasoning/group_1.png

Primary Goals

As Lucid gradually made these data imports available to customers, customers who used these solutions had drastically different needs than prior Lucid customers. Through numerous rounds of research calls (which I typically co-ran with a PM) across different industries, consistent needs emerged:

  • Create: save time drawing diagrams by automatically generating them from data.
  • Analyze: Filter, sort, highlight, and understand the data at various levels of abstraction.
  • Monitor: Quickly understand information at a specific point in time, or over a period of time.
  • Exporting & Control: Use a diagram as an interface to manipulate actual changes to data and affect their original systems of origin.

As we shopped around our experiments and prototypes to customers, their reactions confirmed our assumptions. For example, this Solutions Architect who imported AWS into a diagram:

I appreciate not having to do this by hand, because that’s always fraught with danger of making mistakes. It's much better to import directly from the source. — Lucidchart Customer, 2017

And this engineer in the insurance industry who needed to communicate network security to people who aren't as familiar with cloud infrastructures by simplifying and abstracting data:

I need to show all of my infrastructure from the perspective of security architecture, and the people who see them are not necessarily network specialist. — Lucidchart Customer, 2017

Lucidchart had uncovered a potentially new market which required a set of features we were uniquely poised to build.

Proving the Model with Org Charts, 2018

While Lucidchart's AWS import feature captured our excitement about the future possibilities of data visualization within diagrams, it wasn't the easiest environment for technical validation for two reasons:

  1. AWS has an enormously complex, heterarchical, and unpredictable data structure that can be difficult to express in two dimensions.
  2. AWS practiocioners who create diagrams represent a small and specialized market.

So we pivoted to a broad market with a predictable data structure: Org Charts.

Phase I: Early Discovery of Existing Org Charts Experience

Org Charts are a hierarchical diagram that depicts reporting relationships within a company. At this point, Lucidchart's Org Chart feature had existed for several years. It was mostly built to import simple CSV files to create org charts quickly. While helpful, there were a number of issues with the experience and we turned to our users to understand how we could help.

Compilation of User Research

The Product Manager (PM) and I had been interviewing Org Chart customers over a number of years, so our initial discovery involved digging through old call notes, support tickets, and conversations with Sales and Support. The initial findings were:

  • Org Charts were not performant: our solution could only handle a few hundred shapes, and slowed to a crawl with larger organizations (such as 1,000–10,000 employees)
  • Automatic Org Chart layout was unpredictable and frustrating: adding new shapes to the Org Chart tended to break it, and running the "Clean Up" layout function had seemingly random results
  • Floating UI was intrusive and distracting: the floating modal of settings for the Org Chart frequently got in the way or covered up valuable information.
  • Data could only be imported once: After the initial CSV was imported, there was no way to update the data other than to start with an entirely new diagram. Updates to the diagram couldn't be pushed back to the data source, either.

/lucidchart-visual-reasoning/Untitled%201.png

Competitive Analysis

We reviewed competing industry tools to better understand the market and alternative possible solutions. We evaluated them on a wide range of criteria compared to our own product:

  • Price
  • Perceived Learning Curve
  • Ability to import and refresh data
  • Handling deleted data
  • Sharing views with collaborators
  • Layout Styles
  • Automated Layout
  • Handling photo uploads

This analysis continued to inform and refine our growing list of requirements for a new, successful Org Chart solution.

Phase II: Requirements and Early Prototyping

Based on our understanding of the problem, we outlined a lengthy list of nice-to-have features that would best meet the needs we observed. We prioritized these in a RICE-like score and sought to take the biggest pieces first.

A common complaint was that our Org Charts were simply dated (and let's be honest: ugly). An early priority was to develop a new, modern aesthetic that would present well in slide decks and presentations (a very common use case).

/lucidchart-visual-reasoning/Untitled%202.png

One of the most painful issues we faced was automatic layout. Lucidchart allowed users to move shapes and employees wherever they wanted, then offered a "Clean Up Layout" function to bring it all back into alignment (though usually didn't do a very good job). We hypothesized that users would rather let us do the heavy lifting on the layout so that when one moving one employee among thousands they wouldn't have to readjust their entire diagram.

https://d33wubrfki0l68.cloudfront.net/bc71bcbf0fbe67f5e32d30cd69ca41ca242a2000/166b7/lucidchart-visual-reasoning/kapture_2020-07-28_at_15.29.45.gif

I created this simple prototype in Framer where customers could try dragging shapes and we would ask them what they expected to happen when they highlighted a particular drop-zone. The feedback from these simple tests formed the backbone of what would eventually become our fully automated layout algorithm (seen below).

https://d33wubrfki0l68.cloudfront.net/bc71bcbf0fbe67f5e32d30cd69ca41ca242a2000/166b7/lucidchart-visual-reasoning/kapture_2020-07-28_at_15.29.45.gif

Early prototypes also saw heavy exploration into the Org Chart UI (now moved off of the canvas, and anchored to the left side of the screen). Testing these features was often task-based ("How would you get this to fit on one page?" to prompt them to look for Layout Style) and changed frequently.

/lucidchart-visual-reasoning/Untitled%203.png

Phase III: Alpha & Beta Release

For a few months, while we continued building and iterating, we invited specific customers to participate in our Alpha and Beta programs. These customers could use the new Org Chart functionality while it was still being developed and send us feedback along the way.

/lucidchart-visual-reasoning/Untitled%204.png

We piped these messages into our analytics service and exported them about every week or so. I took the time to organize, tag, label, and sometimes translate any feedback to summarize it for other team members and work the improvements into our upcoming sprints.

/lucidchart-visual-reasoning/Untitled%205.png

As with any updated design, change was hard for some:

This is stupid. previous version was better

The updated features are terrible and hard to navigate

But there were a lot of supporters!

Very Cool! I love the new style - the left panel is so helpful. I love the quick building

Just great and simple

A great work. well done!

always improving - love it!

THIS IS AWESOME - thank you for making life easier.

This constant flow of feedback made the iteration and improvement cycle much easier (and much faster).

Integrations & Data Handling

Lucidchart was stepping into its first role as a data warehouse and all the baggage that comes along with that. I interviewed customers with my Product Manager about data security, compliance, sharing and permissions, bi-directional syncing, and more. Lucidchart was past being a simple Org Chart creation tool, and was now becoming a full-fledged data visualization tool.

Customers often told of the complicated workflows they had developed to create an Org Chart:

"Right now, I download data reports from SAP as Excel files and use R to filter and trim them down. Then, I import into a template in Lucidchart, make changes as needed, and download them as PDF to include in Sharepoint. ... This could save weeks of peoples' time." — Lucidchart Customer, 2019

We were able to drastically shorten the time-to-value by creating integrations with some of the more common tools. This included importing, exporting, filtering, and managing merge conflicts with data refresh. If the concept of merge conflicts sounds familiar, you likely work in software developement (specifically, Git). Most of our Org Chart customers, however, were from non-technical backgrounds and had no concept for the idea of viewing a "diff." Instead, we went through several changes in language such as "change," "delete," or "keep."

/lucidchart-visual-reasoning/Untitled%206.png

We introduced this feature to existing customers and watched them use it. We'd give them a task such as: "Import your organization from Google, then add a new employee to the diagram. Now, refresh the data from Google Sheets—what do you think will happen to that new employee?" (Hint: they would see the above dialog asking them what to do.)

"Oh yep. That worked great! That was everything I was hoping for and envisioning when I was creating an org chart." — Lucidchart Customer, 2019

Seeing as the user flow for refreshing data could be very different depending on the circumstances, I would spend a lot of time at the whiteboard with engineers coming up with diagram like this to best understand how to keep track of the UI logic:

/lucidchart-visual-reasoning/Untitled%207.png

Phase IV: Public Release & Outcomes

My team worked with Product and Engineering Leadership to determine a release date based on some of requirements and what we considered "production-ready." We gradually phased out the old Org Chart functionality (deprecating the old shapes, providing messaging to switch to a new experience, but continuing to support the old functionality where present) and heavily advertised the new experience.

We saw positive signals of success in the early release, but the real results came over time. Like most software companies, we were highly focused on retaining users and making sure their first experience was positive. Compared to some of our other antiquated data features, Org Charts came out on top.

/lucidchart-visual-reasoning/Untitled%208.png

I especially enjoy this graph as it shows when the new Org Chart feature was released (July) and how well it performed over other data and automation features in Lucidchart.

/lucidchart-visual-reasoning/Untitled%209.png

Far fewer Org Chart users needed to use the Help Center compared to other Lucidchart features.

/lucidchart-visual-reasoning/Untitled%2010.png

The Full Visual Reasoning Workflow

Remember those needs we identified from early data and automation users who used our AWS import functionality? This became a use we coined "Visual Reasoning." In case you missed them above, they were to:

  • Create: save time drawing diagrams by automatically generating them from data.
  • Analyze: Filter, sort, highlight, and understand the data at various levels of abstraction.
  • Monitor: Quickly understand information at a specific point in time, or over a period of time.
  • Exporting & Control: Use a diagram as an interface to manipulate actual changes to data and affect their original systems of origin.

Our new Org Chart implementation featured an end-to-end experience that boasted solutions to each of these use cases. Users could create Org Charts automatically by importing from external sources like spreadsheets. They could analyze using filtering, conditional formatting, and formulas. They could monitor by automatically refreshing their data to see what changed over time. Lastly, they could export their entire dataset (or just a set of changes) to implement them in the original data source.

/lucidchart-visual-reasoning/Untitled%2011.png

While there was (and still is) plenty of work to do to flesh out each of these use case, we had designed and built a robust platform that was ready to be scaled to other experiences (remember AWS?).

Cloud Insights: AWS Import v2 (and my move into management)

As Org Charts continued to grow, I moved into a UX management role. I continued to mentor other designers through new features that supported Lucid's data and automation platform. One team that soon emerged was called Cloud Insights. It reprised the work I had done previously to allow Lucidchart to import AWS architecture, but this time it would be built on top of the same technical architecture as our new Org Charts diagrams. Building on this platform meant that it inherited the same benefits as Org Charts: filtering, conditional formatting, etc.

/lucidchart-visual-reasoning/Untitled%2012.png

My work now involved helping designers replicate the same success we had with Org Charts with AWS, Salesforce, and new array of integrations planned for the end of 2020. I assisted designers they delivered new Information Architectures to handle multiple data and automation frameworks, maintained a consistent design system, developed and nurtured a "Trusted User Group" for continuous user feedback, and delivered incremental value to their customer base.

The Lucidchart "Cloud Insights" Add-On was introduced in January 2020 and, as of the time of writing (July 2020), has outpaced financial expectations.