A few days ago, I read an article1 about trends for 2022 in data analytics. One of the opinions paid my attention more than the rest. The thesis was that in 2022 we can observe “the death of predefined dashboards” which sounds odd to me.
Maybe it is only some kind of over-interpretation of what is happening in the industries and an attempt to call it controversially. Nevertheless, decision-makers can take it for granted and start a revolution in organisations harming analytical processes, workflows and widely understood data culture.
Let me touch more deeply on why I bare such an opinion.
The case with data literacy
I would love to see legions of employees who are able to read, interpret and work with data fluently at every level of the company’s hierarchy. But we are not there yet, as all surveys of all consulting companies show us.
For years we have been observing how companies have been putting a large focus on data democratization. The main evidence of that is an evolution towards a data-as-a-service direction by using cloud-based solutions to empower different users in data analytics. However, most of that significant potential can be easily lost just because of the immaturity of the organization’s data culture and the data literacy level of each, single employee.
Frankly speaking, too much focus is on the technology side and too less on people. Companies still mainly invest in training improving technical skills or ability to use specific tools. Training which teaches how to use data for a specific purpose is in minority, even on the market is hard to find such offers. We must remember that employees have different backgrounds and different skills. Some of them would always need assistance in data analytics, just because their core skills are allocated somewhere else and there, they bring business value. We shouldn’t require them to waste their time learning how to work with data, while they should master other skills.
Challenge with an approach data as a product
The next point to cover is how those organisations are advanced in digital transformation. Before introducing a new strategy, some basics must be prepared. Many companies would like to be data-driven, however still suffer from a lack of integrated, automated, and accessible databases that provide high-quality data. And it is not a completed wish list.
Efficient and business valuable data sets serve specific business areas. In most cases, it means that different business areas have data prepared differently including data aggregation, hierarchy, and perspectives. The huge challenge for organizations is to provide an environment, structure, and infrastructure to approach data as a product. It requires investment in hiring an adequate number of professionals and changes in existing processes and technology. Apart from that, DaaP is still a fresh concept and companies need time to get familiar with it and step in on this journey.
Underdevelopment of tech-savvy
I’m writing above about too much focusing on tech training. However, some companies don’t have any vision of how to support their employees in their tech-savvy journey while still expecting results.
I was the victim of such an approach gaining access to the tools without any training and vision of employee development and setting a clear learning path. Worse, I was required to figure out how to upskill myself. That was a horrible experience, both for the employee and the organization that ends up in frustration and lack of results.
Mature organizations employ professionals who take care of the technological development of employees in accordance with the company’s long-term strategy and vision. They make sure that the skill set of employees can shift the company from point A to point B. Without them or similar roles, no major changes can take place.
The hell of multi-sources of truth
If you are a fan of Marvel like me, you know what chaos can be brought by having multi universes. The same risk can be a case when we allow separate business units to use databases without supervision. Business units may report the same metrics differently only because they understand or define them differently. From the inside, we can observe that data retrieval is processed in a different manner.
This generates a bunch of problems. Especially in proofing whose numbers are correct ones, and this requires additional time and resources that could be spent on more valuable tasks. Not to mention ruining trust and mutual relations between departments and employees.
As a key conclusion, I would say that giving employees the freedom to create their own dashboard places a huge responsibility on their shoulders and requires them to have various sets of technical skills. Such a strategy may be similar to throwing the baby out with the bathwater if companies do not invest time and money in ensuring that their employees acquire the skills they need, are equipped with the right tools and data sets can be used without worrying about the disinformation.
It is a chilly morning. I stand in the middle of the kitchen and look at my lovely daughter after our regular morning battle to get her ready for school. Apart from all rage that she carries right now inside, she is like a delicate flower torn by the wind. I ask myself where is the point to force her to get up so early and expose her to all these frustrations that will come for sure today when she tries to remember all useless knowledge. The Polish education system sucks.
My daughter, as the next generation of humans, will face many new challenges in the near future. Climate crises, energy crises, increasing inequality, overpopulation, the collapse of democratic rules … just to name a few. The current education system does not prepare our children for any of the challenges of the 21st century.
Experts agree that for our kids to be able to adapt to the new environment and face what the future will bring, they must master four basic human skills. They are called 4C’s for the 21st century: Critical Thinking, Collaboration, Communication, Creativity. And what is more! According to the experts’, 4C’s are the cornerstone skills learners of all ages need to be successful in life.
What the hell, do these 4C’s have in common with data storytelling?! You would ask. Well, I got an idea for this post asking myself how can I support my daughter in developing 4C’s. Then I asked myself if I was using 4C’s and how beneficial it would be.
4C’s for Future, 4C’s for Today
If you’re wondering where the future starts, the simple answer is today. It doesn’t matter how old are you and what challenges you face in your daily life; these four skills definitely help you achieve more in less time.
Critical Thinking – foremostly
In the past century, people have struggled with collecting and obtaining data for their studies. We are now reaching the point where anyone with access to the web has access to a large amount of data and can do their own analysis. Data democratization, like everything else, has two sides of a coin. Unfortunately, the dark side of the common usage of data is to mislead people and create fake insights.
I love the TV series “Ancient Aliens” but the level at which they treat and interpret scientific facts is very innovative – gently speaking. For me, it is a piece of good entertainment, but we can imagine how that trivial approach to science and what is worse mass-broadcasting this approach, can implicate damage in some people understanding of ancient history without questioning that “revealed truth”.
Critical thinking has its roots in curiosity. Before you judge or draw a conclusion based on information, you should dig deeper to make sure that your conclusion is not skewed by shallow analysis or dubious data. Similarly, to “Ancient Aliens” you can create the most breath-taking story about your discoveries, but where is a meaningful value from this fairy tale?
Critical thinking is a habit of questioning others and yourself and the good news is that everyone can learn it. To develop this habit:
1.Ask the right questions and validate your own logic.
“There are no stupid questions!”. I hope that you’ve heard that many times. If you haven’t – change organization! Asking questions is the simplest and the best way to verify your or others reasoning. Use the below questions to warm up your critical thinking: “Where data came from? Do I trust data sources?” “What is data quality? Are there any missing entries?” “Does the data sample is big enough? Does it present only a small part of the bigger picture?” “Do all factors are included in the analysis?” “What business questions does this analysis cover?” “Do I not overcomplicate things?”
2. Deal with your (or others) biases. Remember we too often look for evidence that supports our prior beliefs.
All of us have some kind of the burden of biases. It strongly affects how our brain interprets information and draws conclusions. Studies show that we have a natural tendency for ensuring that we already believe. That tendency can be very harmful to the recommendations which we provide. To understand better how our biases play tricks on us read the book “Mindware. Tools for smart thinking” Richard E. Nisbett.
3. Take time to evaluate the topic from different sides and seek diversity.
Most of the time we are in rush and that hurt our reasoning and the quality of work we deliver. So, hold your horses and invest time in finding out other people opinions. One question about “What causes revenue decline” can have multiple answers depending on the point of view. These points of view can be very valuable and let you create a story with a wider spectrum.
The self-made man is a myth. No one is one hundred percent accountable for his/her success or failure. We are the result of many factors like genetics heritage, family relations, culture constraints, environmental influences, and life experiences. All together constantly have a huge impact on how we perceive ourselves and make sense of what surrounds us.
Have you read a biography of Bill Gates? Bill Gates maybe wouldn’t be so successful in his field without a few coincidences like exposure to the computers in the earlies ’70s as a teenager (what kid had that opportunity!) and mother who served in IBM board and helped in securing his first big deal with IBM. Of course, he used those opportunities very well, but would he have been the same Bill Gates without those chances?
We as humans operate in tribes. Without other members, we wouldn’t survive. If you want to be successful in your life collaborate with other people and leverage their skills and knowledge, especially because domain specialization is so deep that it is hard to be a Leonardo da Vinci in the 21st century.
Some people find it easier to collaborate with others, others find it harder. And again, self-discipline and practice can help you develop habits:
1. Invite subject matter experts to discuss and review your data, analysis outcomes, recommendations. They can bring a new fresh outlook to the table and create together with you more valuable insights.
2. Ask other analysts how they would approach the analysis of particular datasets. Maybe they did something similar in the past and you can save plenty of time.
3. Gather as much information as possible from stakeholders to focus on what matters for them instead of waist time on general questions and findings.
No other animal has developed communication skills like humans. We wouldn’t be able to conquest the whole planet without that one unique skill. Due to that skill, we can build strong relationships inside our tribe and with other tribes, convey abstract ideas and pass on incredible stories about faraway lands.
Good communication starts with a good strategy. How many times have you failed to convince others even though you have done an excellent analysis and prepared actionable recommendations? Your message didn’t get through because it wasn’t appealing to them. Consider the below points and tailor your message to be more impactful with your audience:
Ask yourself what are the main pain points for your audience?
Are they data literal and how advanced?
Are they subject matter experts or do they need more introduction?
What can they expect from you? Raw analysis with insights or clear guidelines and scenarios with recommendations?
I’m not a fan of getting too creative in the visual representation of data. Data visualization is already an abstract form and making it more complicated by adding non-intuitive graphic shapes does not make it better. However, using creativity to look at a problem from a new perspective and consider new possibilities is a direction every data storyteller should take. Most of the time we stay within our standard thoughts or typical suspects. This leads us in the long run to
However, using creativity to look at a problem from a new perspective and consider new possibilities is a direction every data storyteller should take. Most of the time we stay within our standard thoughts or typical suspects. This leads us in the long run to intellectual castration, which has several serious consequences, such as missed opportunities for the organization, unrecognized in time threats, and a retreat in development. Creativity is again a skill that can be acquired and mastered. Experts recommend the following exercises to strengthen it:
1. Learn from others and surround with inspiration The more you collaborate with others, read a lot, and learn new things, the more creative you are. You need to have enough information gathered to connect the dots and then new ideas start appearing.
2. Enjoy what you do Doing things with passion produces unexpected outcomes. You need to be truly dedicated to your work to be able to find new solutions or patterns. If you do not like what you do, you are not involved and interested, do not expect from yourself outstanding performance. Maybe it is high time to change profession?
3. Findtime to do nothing Give your brain a break. My best ideas show up mostly when I do something different like taking shower, doing exercises, drawing, or reading. When you feel overwhelmed, simply switch your activity, and focus on something else. Your brain anyway still processing that idea in the background and doing the magic.
4. Walk Stanford study has shown that walking improves creativity. So, when you have a problem, simply take a dog for a walk. Many CEOs already have introduced walking meetings within their organizations to increase people ability to think out of the box.
5. Hypothesize One of my co-workers taught me a great technique. It is a simple question to ask, “What would have to happen to achieve XYZ”. That simple technique removes any barriers from our brains and shifts from concentrating on constraints, what we naturally do, to focusing on possibilities.
The ethical approach to data visualization has many faces. One of them is dealing with missing data and the way of communicating them to the audience. In the real world, we face situations that our databases are incomplete. This is a common case of many reasons. Some are technical errors that can occur during ETL processes, others appear when data is collected manually, especially as a result of surveys, as people often fail to answer all questions.
Statistical procedures often eliminate entire records when only one variable is missing. This leads to a dramatic shortage of statistical samples. However, many times, even though our data is leaky like Swiss cheese, we have to present them and what is even worse, draw conclusions, because having 100% of data is in many cases ineffective and unrealistic in terms of costs and time.
To stay honest with our audience and to present the observations or phenomenon to them in the most transparent way, we have only two options: to present gaps in the data or imputed data in place of missing data. There are several imputation methods widely used in statistics and statistic data modelling. The most common ones are:
Case deletion – omitting cases with incomplete data and not take them to analysis.
Zero-filling – imputation of value 0 for all missing data.
Linear interpolation – replacing missing data with estimated values.
Marginal means – the mean value of variable is used instead of missing one.
More explanations of the specific methods you can find here.
Nevertheless, what method we are going to use, we need to communicate to the audience about which data comes from observations and which ones are imputed. This communication should be given in voice and visual form to strengthen the message leave no room for presumptions.
Dilemma – show gaps or imputed data?
Many strategic decisions are data-driven and missing data impacts the overall understanding, interpretation and reasoning of a phenomenon if not properly addressed.
Recently I found interesting research by Hayeong Song and Danielle Albers Szafir that shed some light on how we visually communicate missing data, which has a significant influence on data quality perception and on confidence in drawing conclusions. Research emphasizes that visualizations that highlight missing data but do not break visual continuity are perceived by responders as those with higher data quality. The general conclusion is that imputation methods are better graphical choices than simply removal of information as they do not decrease perceived data quality as much that have consequences in the decision-making process. However, the very important aspect is to highlight imputed data by different shapes or colours. Another interesting graphical decision is to present imputed data as error bars. It gives our audience additional information about the likely range of values.
The research results in Figure 5 (b) clearly show that linear interpolation has the greatest positive impact on the perceived quality and accuracy of the data, and the visualization with data absent (Figure 4 (a)) is the lowest.
The research was carried out for two commonly known visualization: a line chart and a bar chart. Both graphical choices gave similar outcomes.
I have several books which are like a shining star that guides me through the darkness. One of them is “The Little Prince” Antoine de Saint-Exupery and quote from that book: “You become responsible, forever, for what you have tamed”. I believe we should have exactly the same approach to our analyses and their graphical representations as data analysts or data storytellers.
On September 26, 1983, in the middle of the Cold war, Russian lieutenant Stanislav Petrov was on duty at the command centre of the nuclear early-warning system. The system reported that six missiles were fired from the US toward the ZSSR. Petrov based on provided information had to decide whether the alarm was true or false and to obey or not obey orders. After countless minutes that seemed to be an eternity, Petrov judged that it was a false alarm and saved the world against third war – the nuclear for sure. Later, the investigation revealed that the system malfunctioned.
But what kind of the world could we live in now if Petrov had not considered other options of the system’s response? Having that historical event in mind, can we trust any information without a doubt?
As data analysts or data storytellers, we are like a nuclear early-warning system. We provide people with the information they need to make critical decisions and shape the future. It is a very responsible role.
Why is so hard not to lie with data?
Does it sound controversial? I believe so. Does it sound realistic? For sure. Why do I think so? Are you confident that you know all aspects of a subject that you want to present to others? Have you considered all possible options and looked at them from all involved stakeholders’ perspectives? Are you sure that the data set time period is long enough, and data quality is high? There are more questions than answers. So, tell me which version of the truth you are holding in your visualizations?
I do not accuse anybody to mislead people on purpose. Most of the time when we prepare data analysis and data visualizations to communicate information, we have pure intentions. The case is that we hold some biases and believes, and our brain uses previous experiences, and constantly makes unconscious assumptions. All that influences our thoughts and perception.
Harmful data visualization
Let’s do the mental exercise and think together about how harmful data visualization can be. Currently, I’m reading an exciting book by one of the most recognizable authors of the information visualization domain Alberto Cairo “How charts lie”. In one of the chapters, there is a story about nationalist Dylann Roof, who killed several Afro-Americans by being influenced by some charts that presented a number of crimes vs ethnic roots. That shocked me and opened my eyes to the potential consequences of distributing misleading visual representations of data.
That warning is more for data journalists and other people who juggle with data publicly. Often to get more votes or support or to influence some kind of the audience line of thinking. However, even in the business environment, we must be cautious not to make the same mistakes, because results can be catastrophic and have a real impact on people. Nevertheless, all of us should remember that when we share any data on social media or on other web pages.
The potential negative impact of wrongly done analysis and poor data visualizations:
Hundreds of people can lose their job,
Profitable business sector can be shut down,
Launch of a new product can miss the target,
Thousands or billions of people can be at threat because of the release of the new drug.
This vulnerability is real because people who make decisions make history. There is always a human factor in any success or failure.
Do you feel like an influencer?
Some time ago I had a lot of fun preparing and sharing data visualization. But currently, I’m not so eager to do that. I didn’t have enough confidence in the data that are available, and I don’t have enough time to dive into and understand the specific subject, make analyses and investigations.
In upcoming posts, I’ll focus on ethics from a data visualizations point of view. The first one is data range.
Insights could differ very much in case of changing data scope. Anyone who has some shares on the stock market knows that depending on the selected time range he or she can observe positive or negative trends. The same cognitive dissonance we can have presenting data within our organization. Maybe in the last two years, we achieved tremendous revenue growth, but looking at revenue from a longer perspective, it can turn out that we even got closer to the results from the financial crisis (pick your favourite one as an example, they come and go periodically).
Figure 1 depicts what kind of understanding and feeling the investor can have to look at the same data but from different ranges. The left chart can indicate that results are declining, but when we look at the right one, we can see that in the longer perspective trend is positive.
Of course, our narration can be built around the latest two years of growth, but we shouldn’t hide information from the bigger picture. The approach in such a case should be to display the bigger picture first – a longer period of data is displayed and then zoom in on the last two years to present factors of recent revenue growth.
Another example, which is notoriously used to present voting results, is presenting people support for particular parties but having only people who voted as the full population. When I listen to the news in the mass media, often people refer to the election results without considering the voter turnout. That narration skews reality. Let’s see the below example. Figure 2 shows the result of the latest presidential elections in Poland. What will most people remember from the chart? That Duda won and had more than 50% of public support.
But this is not true! The real public support for Duda was 34.49% if we consider the voter turnout. The voter turnout in this election was 68.18%. It means that 31.82% of Poles didn’t go on the election. I would love to see in the mass media charts which present the entire election results, including those who didn’t vote. Then we would have the complete picture of people’s political preferences. However, I still see truncated data scope.
By manipulating data range as a timeline or included/excluded categories, we tell different stories about data and evoke different understandings and feelings in our audience about the subject. Let’s remember that to not lose in translation the most objective view possible.
I promised to prepare a post about strategies of BI product successful adoption. It is a really hard work to achieve this. Not because your company CEO is a miser, and he or she doesn’t want to give any penny more for technology or on hiring a new workforce or outside company that works for you. The true challenge is to change the way people think … and behave.
Is Excel still the main data processing tool in your company? Do people still value working with this tool, because of its simplicity? If your answers are yes, you should already feel that changing their work habits is not a piece of cake.
BI solutions are new tools that need to be adapted in your organizational structures with proper care. Introducing a new tool goes hand in hand with introducing a new process. Introducing a new process involves managing change. And that is exactly what adoption is – the change management case.
Many organizations have in their structures Change Management department that can support BI projects in better and faster implementation by leverage knowledge of change management processes and techniques. Human Resources department can be very useful as well when it comes to redesigning some people habits and behaviours. I highly recommend asking them for support in any initiatives involving the introducing any new solutions.
Before we delve into the subject, let me briefly explain what change management is. It is a structured approach to prepare and support the entire organization and individuals in making organizational change.
For me, the term “change journey” is more appealing than “change management”. I associate change with the human factor more than with processes because without people’s willingness any change will take place. There are several methods or frameworks to lead successful change, however, for BI products adoption I found ADKAR modelappropriate.
A like AWARNESS of the need for change
From my experience, it is very important to start communicating about the change a long time before it happens. There is a psychological explanation behind this: people don’t like changes. They must get familiar with it, so preparation is key.
There are many channels that can be used for that purpose such as intranet, emails, workshops, and face to face meetings. The message should focus on answering why change is needed and on the benefits for each individual and the entire organization. It is important is to address any concerns or biases related to the change. (I wrote more about it here).
Ask HR department for support in this sensitive case. Involve top management as the voice of change.
D like DESIRE to participate and support the change
Although all efforts go into Awareness phase, it doesn’t mean that the results will be spectacular. The reason is that each person must make their own inner decision whether to support the change or not. Many practitioners point out that win hearts and minds is the most difficult part.
The main challenge here is how to get people to care about something they don’t care right now?
As unfaithful Tomas, most of us have to see to believe. Data platform projects are relatively long-term and for most of the time, end-users do not see results. Fortunately, we often create PoCs (proof of concepts) or prototypes to test certain assumptions. These small pieces of work can be shared to prove major concepts of a new approach. If this prototype is prepared to address one of the main company’s pain points, it would be easier to promote the new approach in the organization because of its undoubted value, which shows how this change can work for them.
K as KNOWLEDGE on how to change
This phase is associated with learning new tools and new skills. Many organizations use Excel to communicate data. Most of the time they prepare reports and send pdf files by email. Introducing a new tool like Power BI or Tableau forcing breaking old habits and behaviors and building a new one. This transition must be supported by delivering inhouse training that will bridge the gap between current knowledge and skills and desired one. In addition, all training must follow with creating an internal space where people have access to information about this new tool and have a place where they can share their experience and find answers to their questions.
Too often I observe a common scenario, that a new tool is introduced, however, staff training is not budgeted. This gives rise to a lot of frustration when people are required to provide valuable analysis, but they lack skills.
A as ABILITY to implement desired skills and behaviors
Having knowledge doesn’t mean that you know how to put it in practice. It takes time for people to develop a strong conviction that they are capable to use new tools for expected results. They won’t do it without support from the company side. Bringing in trainers or field experts who will work with them for a while can speed up learning process and smooth transition from the old to new approach. The main slogan here is practice, practice and even more practice.
R as Reinforcement to sustain the change
Have you heard about the “JoJo effect” when it comes to weight loss? It often happens that people who put a great effort into losing a few kilograms and spent several weeks or months on exhausting diet and psychical activities, very quickly regain their original weight. The reason is that they didn’t change their habits but only suspended for a while. There is even scientific proof that our brain reverts to safe, comfort and well-known practices. Therefore, maintaining the change is very demanding.
Before we are going to introduce a new approach, we must find out how the current processes are like and what people think and feel about it. Most of cases in organizations there are two or even more ways people do certain things. The first one is official procedure which can be found in organizational documents or regulations. The second one is the informal way people really work. This informal approach manifests their habits, behaviors and beliefs and is significant for us. Without revealing true processes, the new change won’t be successfully implemented due to lack of knowledge of how to implement it in such a way that people would be open to accept it.
You don’t have to start big. Start small.
When working with the client, we usually choose only one business area to improve. This could be sales performance, for example. Then we makeover reports, or we design them from scratch, develop and make them available as the reporting platform. This short cycle has many benefits. First of all, we can quickly verify technical aspects of the proposed solution, check with the stakeholders whether the product meets all the requirements, and what is most valuable if the product can be release to wider audience and prove its usefulness to them.
Leverage old tools
Instead of introducing rapid change as revolution, sometimes we can achieve better results by doing it in slower pace like evolution. If your employees are used to using Excel, don’t take it away from them. Most of the BI products have possibility to extract data into an Excel file. Focus in the first phase on process automation and ensuring a single source of truth. Anyway, they have to use the BI product to retrieve some data. Over time, as they trust and become familiar with the tool, they will start using it instead of extracting data from it.
Top management involvement
Recognition and a pat on the shoulder is not enough. Every change (as well as every initiative) requires fully committed top-level managers.
Several years ago, at one of my previous employers, I was involved in designing and implementing a new business intelligence tool. The goal was to provide a large number of reports covering all business aspects. The task wasn’t easy due to its complexity and data accesses challenges. Most of data were stored with IT department which didn’t want to share accesses. The first release took us almost a year (it was long before I heard about Scrum 😊). As you can imagine tremendous effort and time has been invested in delivering this tool.
This project was under company digitalization umbrella and aiming to improve the availability of information at every level of organizational hierarchy. However, most senior managers didn’t use this new platform, where they had all important information at their fingertips. They preferred the old-fashion style to send tones of emails asking for these essentials.
As you get the impression the adoption wasn’t spectacular, I would say that we missed the momentum.
There is a proverb that “the example comes from above”. I believe that if senior managers presented themselves as hard users of the platform, it would have enormous impact on the platform usage.
Ambassadors of the new approach on each level of organizational hierarchy
Apart from Top management, you need army of true believers, who will be a voice of change. These people should come from different departments and from different levels of company’s hierarchy. They should be a role model for their colleagues.
There is no better option to involve people by giving them the chance to become fathers and mothers of the initiative. Parents love their children selflessly.
You can follow the tactic of one of my clients. They formed working teams with people from different departments, who were involved in the design of a brand-new reporting platform. These people talk about their new project in the halls, canteens, and during cigarette breaks. This is a perfect example of viral marketing!
Support, support and once again support
How would you perform driving a car without hours of training and a good teacher? Likewise, your people need teachers and resources to learn and master their skills. You can leverage whatever works: on-demand or instructor-led training, online resources, community groups or newsletters with examples how to use and read data from the new BI product.
One of my clients constantly uses emails to send out extensive examples presenting usefulness of the BI product. They provide screenshots and guide others on how to use a tool, but more importantly how to analyze with the tool and create insights.
Start with day one
The last good practice that I want to present is to combine BI products into internal processes. This tactic forces people to use this tool and cut any discussion, whether they deem it relevant or not.
That tactic is for companies that really have ambitions to become a data-driven companies quickly. In such case all teams have to start workday by checking the latest data and on that basis and making decisions what they will do today to improve the performance, for example.
The great example is Daily Scrum – meeting (one of Scrum time boxes). During this event, a team relies on yesterday’s activities planning today’s activities. They use Kanban board to track data about the progress of current work.
Likewise, dashboards or reports should be used as a mandatory tool for daily stand-ups to discuss ongoing performance and set the next directions.
Have you ever thought that it is possible to discriminate people through data visualization design? Several years ago, it sounded strange to me too, but indeed, it can be done unconsciously if you are not aware of the topic.
Discrimination is most often associated with skin colour, gender, age, religious beliefs, or nationality. However, this negative social phenomenon can have much broader spectrum. One of them, not at all intuitive, is data visualizations practices. The topic is gaining importance as more and more data is used to explain global processes, and those with difficulties in that area are being left behind. It may not be simple, but the onus is on data community and data visualization practitioners to develop new best practices to communicate data in more democratic way with those with difficulties in this area in mind.
To make data visualization more accessible to a wider audience, three dimensions can be improved: vision, cognitive and learning difficulties, and motor capabilities. The basic, obvious difficulty is related with vision impairments; but the degree of impairment is key. I will not discuss the most severe degree, which is blindness (this is a topic for different post), but I will bring closer the subject of colour-blindness and low vision impairments.
In data visualization, colour is the most important communication channel. The ability to see and understand the meaning of colours helped our ancestors to survive in deep jungles or on savannas. Colour informed them about non-toxic food or allowed them to spot predators in the forest.
Today, we are still sensitive to colours and these naturals reactions are used in many ways. For instance, most warning signals use red colour, because we naturally associate it with danger or action (red is a colour of the blood). Studies show that prolonged exposure to the red colour can cause the heart rate to accelerate as a result of activating the “fight or flight” instinct. In opposite, blue colour has a calming effect.
However, not everyone can see colours. Approximately about 10% of human population has trouble seeing colours correctly. If you would like to deepen your knowledge about types of colour-blindness, please check the website. There you can learn about causes of colour-blindness, test yourself, and find a tool to check if prepared visualization is in line with best practices.
There are several basic principles that improve your colour palette and enable visualization for broader audience. To understand them we need to understand two important colour properties: hue, and saturation. Hue defines colour in terms of pink, blue, yellow, or magenta. Saturation is nothing more than volume of the colour. By juggling these main properties we can improve or worsen results of our work.
First of all, stop using red-green palette which is confusing or even unrecognizable to colour-blinded people. This is my humble recommendation. For most people with colour difficulties this red and green colour look the same (see Picture 1).
Most modern data visualization tools, such as Tableau or Power BI already have available colour palettes that handle with the topic. Both mentioned tools have also option to create custom compositions and upload them to the application (custom colour palettes for Tableau and Power BI).
If you are wondering about the right colour palettes, check out the ones presented on Picture 2 and Picture 3. They are nice, clean, and fancy and will work for any reports.
CONFUSING COLOUR PAIRS
Even though we try to avoid the red-green colour range there are still other pairs that resulted in similar way. In recent years I have been observing the dizzying career of the grey-blue duet. I like this combination as well, however, it is essential to match them wisely (see Picture 4).
Sometimes the best option is to simply stick with one colour and play with its saturation to differentiate specific categories or data ranges (see Picture 5). This approach can be used in most visualizations.
More practical colour ranges you can find here, and if you would like to test your composition on specific charts use this website.
Another interesting channel we can use to help visually impaired people easily distinguish between coded data is to assign shapes to different data categories. A good example of how the introduction of shapes can make difference is the well-known RAG.
RAG stands for RED-AMBER-GREEN and is widely used in business environment to communicate performances, risks or statuses of activities. It is most commonly used in project management to report status of tasks, but due to its simplicity, it is also used in data visualization to highlight for instance KPIs (key performance indicators) performance. Red indicates about underperforming, amber that something is an issue and needs to be monitored, and green that is fine.
But as you already know RED-GREEN can be very confusing for colour-blind people. So, my suggestion is to use a shape as another visual communication channel to make sure everyone is on the same page. Instead of format with coloured background, it would be better to introduce icons that have different shape and are coloured in red, amber, or green (see Picture 6).
But what about charts like line chart or bar chart? How can we improve distinction between specific lines or bars? We can use different patterns to distinguish one bar from the rest one or to present several lines on one chart (see Picture 7).
Written descriptions, recommendations or insights can be tricky. Especially when you want to use colour names to emphasise certain points, data categories or issues. How someone, who does not see green colour (see Picture 1) can understand a message “All departments represented by green bars have exceeded their sales targets this year”? This message must be rewritten to “Departments A,B, and C have exceeded their sales targets this year” to ensure that all stakeholders understand it.
In addition to the most recognizable challenge, which is colour blindness in data visualizations design, there is another related to vision loss due to age, accidents, or genetics. For those who suffers from low vision, we must remember that size and contrast of displayed text matters. Especially when we display some materials on screens in conference rooms, but even when you present something via communicators as Teams, or Zoom, size matters. You can read more about the topic here.
When it comes to the font size, there is no one good recommendation. It depends on the purpose. If you are going to display materials at a conference in a large conference room, it is better not to use smaller fonts than 18 when describing axes or legend and have less information on the slides. There is nothing wrong in having more readable slides rather than fewer but cluttered.
A different approach can be taken when creating reports. I would say use a font size 9 or 10 for axis or legend description, but in no other case should you go lower than 12. In reports crucial thing is to group information together or to display them in close proximity to make it easier to interpret or make decisions. That is why optimizing space is so important. These screens can always be enlarged, and anyone can take advantage of them.
The general rule is to maintain high contrast between background and foreground (e.g. white – black, black – white). A typical accessible barrier for people with low contrast sensitivity is grey text or figures on a light background. However, for some people better combination is with lower contrast, because they suffer from the bright background (e.g. they have to change a screen background to the darker to be able read what is on the screen).
As you can see there is no single best answer how to approach this challenge. A good practice is to give people the option to change the display mode from bright (light background and dark foreground) to dark one (dark background and light foreground).
By these small changes, we are bringing better user experience in our organizations or widely, if we prepare data visualizations for the media or other public usage.
It was several years ago. I was in Cracow and, I made an appointment with my friends in a nice vegetarian restaurant. I took with me my nine-year-old daughter. To get there we took a tram. My daughter was very excited about the event and, as a typical child her age, the minute we entered the tram, she started asking where we were getting off.
Fortunately, we sat down just next to an information board, where all the tram stops were displayed. So, I told her stop’s name, pointed to the board, explained to her how it works and proposed counting the stops on her own. I didn’t pay too much attention to it because Cracow is my hometown, and I knew it wasn’t far.
What a surprise it was to hear: “Mom, we’re on the 12th stop”. Knowing it cannot be right (the right number was 3rd), I asked her to count them again, but the response was the same. This situation repeated a few times till eventually she exploded and yelled at me. I swiftly considered the hypothesis that she must have been swapped in the hospital (obviously my own child would be smart enough to correctly count to 5!) and rejected it. Finally, I looked at the information board, and everything became clear.
The culprit of this confusion was interpretation of the information board with tram stops. You see, western civilization uses the left-to-right reading pattern, so this reading order seems natural to us. Linguistic and reading patterns affect reasoning of time and space, as well as relations between these two dimensions. My daughter made a subconscious assumption that the tram stops on the board were displayed in the “normal” order. Her assumption was totally right.
However, it displayed stops according to tram moving direction (right to left) but not with alignment of left to right perception of time (unexpecting design choice). Even though it consisted of a direction arrow, names of stops, and a moving ball pointing to the current stop, my daughter’s brain was still searching for the familiar left-to-right pattern.
And that is why her answer was 12th (count from the left side the stop marked with yellow circle!, but the start of the trip is on the right side)
This story is a great example how people consume information embedded in the time and how they expect it to be displayed. It’s worth remembering that, in our culture, information is read from left to right and from top to bottom. When we work on reports, dashboards or any data visuals, the human brain uses built-in patterns, helping to store information and save energy. Following this simple rule significantly improves the user experience.
Check out my other posts about importance of the time orientation in data visualization:
In this article, I will summary seven common people biases that I observe when dealing with introducing a new reporting system.
Firstly, we need to understand why there is so much resistance in embracing a new? Most people are afraid that they don’t have adequate skills to use new tools, or they won’t easily understand the content of those tools. In some cases, the root of incomprehension is a lack of comprehensive information on why a new process is introduced.
Suppose we want to carry out a successful transition from the old process to a new one. In that case, it’s crucial to understand and address the specific fears and biases of our employees and manage those emotions. Change management strategies should be built around addressing people fears, untrust and incomprehension. Not without significance is a group of employees. Those strategies must differ when you approach data people and non-data people.
But before building strategies that help overcome those challenges, let’s name them.
The issue with data literacy
Data democratisation has enormous potential to change how we work and how we think. A great example is the work of Hans Rosling, a Swedish professor, who, thanks to exposing an audience to data, was able to influence their perspective about how positively the world change during the XX century. However, he didn’t leave the audience without assistance to understand and consume the data. When we give access to data for a wide range of people, we have to remember that among the employees’ population are individuals with various disabilities like dyslexia, dyscalculia, colour blindness or any other cognitive deficits. If we do not consider and address their limitation, our ambitious plan to empower each employee with data will fall.
The technology barriers
The biggest mistake is to assume that for other people, new technology is as intuitive and easy to assimilate as for us (it is a shared conviction among some people who already use some technology). However, the reality is quite different. People do not assimilate technology at the same pace. Some of them need more time and more assistance to become familiar and feel comfortable with new tools. Employees in our organisation have different educational background, age and skills that influence the understanding of any new technology.
The fear of an incomprehension
The result of neglecting issues with data literacy and technology barriers is that people will not use the new tool. They will not build a firm conviction that they can analyse data in a meaningful way and create business insights. For those who already work with data, BI platforms can be seen as data Eldorado. However, for non-data people, the same tool can turn into a nightmare. For data people, It is obvious that we live in the data ocean, and well-prepared data can enrich any process within an organisation or in private life. But not everyone has an analytical mind, and interpretation of data can be a challenge.
The fear of being redundant
Data people often see technology as a threat to current jobs. And this bias has strong evidence in factories and back-offices. If we automated the work of three people, who had done it for 80% of their time, they can feel anxious about their role in the organisation and be reluctant to use new tools. They can even present a negative attitude to a new device or process, explaining its unreliability. However, we need to remember that automation in the BI field is a blessing because it gives employees space to release their potential. In analytics, about 80% of the time is taken on data transformation jobs. By automating this part, people can have more time to explore data and create thoughtful insights and recommendations.
In here, we have reached another bias connected with someone’s skills. Even for people who have the potential, it can be hard to switch from one role to another in a short period. The organisations that appreciate human resources’ value are willing to offer upskilling training, which prepares for new positions, often more demanding. If well-designed and well-performed, this transition can be an excellent opportunity to grow, both for organisations and individual employees. More and more data requires more and more professionals who know how to take out the best insights and communicate them. In the future will be lower demand for data analyst but higher for data storytellers. That shift will be towards a better understanding of the business environment, business constraints, and connecting all those dots into one thoughtful piece of information.
The fear of being seen
BI platforms with updated on daily basis reports give enormous performance transparency in all fields. It enables monitoring people performance more outstanding than ever before. No one will hide. The dark side of this transparency can manifest itself in increasing stress experienced in the workplace by employees. People performance reports should be carefully designed to underpin organisational culture. Depending on the competitive culture or cooperative, the approach to data narration would differ. If you care to have a solid and effective team, your latest goal is to emphasise individual performance. Numbers in such case should reflect the team capabilities and contribution of individual members but not a comparison that can lead to competition within the team. Even in a competitive culture, data narration can have a positive or negative impact. When our objective is to attain goals in the longer time lag, we will desire to evoke positive employee motivation. Positive motivation is a reward for good performance, negative – pain avoidance. A good practice is to research before developing any reports to check what data visualisations bring what emotions and how they resonate with the audience. The red colour is a good example. For instance, at one company where I worked as a consultant, employees decided that red colour on reports negatively impacted their performance and well-being. The red colour was used to emphasise the sales budget realisation, which was under the threshold and shinned in red most of the time during the month. In such a case, another option is to highlight budget realisation and compare it with timeline, or simply change perspective and communicate current goal attainment (in green or blue).
The fear of responsibility
If you are thinking about creating an organisational culture based on individual employees responsibility and engagement, there is no better way than democratise data. The idea behind data democratisation is to democratise the decision-making process as well. However, on the other hand, as a side effect, we delegate more responsibility to the employees in lower positions. Some of them are already capable and just looking forward to that chance; some would be more reluctant. Nevertheless, the essential purpose of data democratisation is to equip employees with a tool that gives them authority to impact their performance significantly and, through this, on the entire company. Another aspect worth mention is reducing micromanagement that negatively influences employees’ efficiency, self-esteem and increases frustration. Having BI tools in place, employees are welcome to use them more frequently in self-management and drive their actions.
The issue with trust
Before we release a new BI platform, we need to be 100% sure of data accuracy. As an old proverb says, it is easy to lose credibility but extremely hard to regain. Data platforms are no exception in this case. They need to have a status of a single source of truth and be irrefutable. Otherwise, the audience won’t rely on provided data and go back to old, common practice. Establish a single source of truth and one dictionary for all measures is crucial. After the release of the BI platform, all other data resources need to be withdrawn for everyday use not to mislead users and not create a parallel reporting system with alternative truth. The massive challenge in this field is to convince people. In addition, shared access to guides and other materials which easily explain how the solution works, data is prepared, and measures are calculated can help people trust the new tool.
In the next article, I will go deeper into practical strategies that are available and easy to introduce in all types of organisations. So stay tuned.
Sometimes it is tough to keep calm and be professional. Especially when your project stuck in the middle and what is worse, you know how to get out of this situation, but you don’t have any authority to move further.
Early afternoon. The call started. You are the only person on the call; however, several people should be already in. You are waiting and wondering if they will show up or not like on several latest meetings. They accepted the invitation, but it doesn’t make any difference any longer. Minutes are passing by, I’m still waiting, and I’m trying to find some positives not to have a feeling that I’m totally wasting my time. Recently, I’m working on naming my emotions, so I’m taking that advantage and start. The frustration wouldn’t be the best description. There are much more picturesque feelings to describe my state like: disappointed, disrespected and unimportance. All of this makes me annoyed.
To achieve success in BI project, the commitment from all parties is needed. Especially from the Product Owner side. The Product Owner is a person who is the mind and the heart of the end-vision of the product. He or she knows exactly what features and characteristics the product must and must not have. In other words, you can not run the project without this Very Important Person.
There are many things which can be frustrating as lack of data availability, weak engagement of IT department or infrastructure with huge technological debt. But those are issues, which proper approached, can be solved. Missing and uncommitted Product Owner is a real threat to the project succeed.
So the question is how to influence Product Owner commitment? I won’t deal with that question. I’m not a psychologist. However, there are several strategies which can be used to find a solution from that impasse.
Change the Product Owner
That is the best strategy. In internal projects, people who are the Product Owners in major cases are the same who have “business ownership”. In other words, they are already busy. They don’t have room to take another responsibility on them. Simply as it is. Frankly speaking, within an organization, there are other people who can be a successful Project Owner for your project. If you have the opportunity to switch the Product Owner, do it. From different reasons, you don’t have to remove the former Product Owner from the project. That person will be a precious stakeholder.
Treat the Product Owner as a Stakeholder
By the way, talking about stakeholders. If there is no chance to change the Product Owner, it is good to set boundaries regarding what kind of decisions she or he has to make. Leave for them only the most important, the most critical decisions. The rest of decisiveness makes within the development team. This approach will reduce the waiting time for approvals and other blocks. Of course, don’t forget to establish those boundaries with the Product Owner.
Find a Subject Matter Expert
If the Product Owner doesn’t have time to help you in designing a product, who else can help? Who can help you prioritize backlog items, answer all development team questions, make those numerous decisions and always support you? It can be someone who works closely with the Product Owner and well-knows the business field in details. And what is more, has time to be present within a project. Another advantage is that in the majority of cases, those people are the end-users of the product, so they really know what they need.
Last but not least. Don’t forget that it is only a job. Take a few deep breaths and after working day, engage yourself in more important things ;).
Purpose of my work is oriented on preparing the best BI (business intelligence) tools to support achieving goals within the organization. In the last post, I described four initial points to start the KYC process. Today I’m going to focus on next steps. In this post, I’m going to share my experience and my best practices on how to gather requirements effectively.
I have seen many guides about what kind of questions is good to ask during business or data analysis. Most of them are useful and relative to the topic. However, my role doesn’t focus only on business or data analysis. The subject is much wider. Daily, I’m using techniques and methods from UX & UI design (user experience and user interface) and data visualisation to create the state-of-the-art BI product. In here I would like to clarify one thing I’m not a BI Architect, who is responsible for the back-end. My research and tasks focus on overall business needs and front-end outcomes. I create content and the “look and feel” experience.
So, you already gathered as much as possible information about the customer company, industry challenges and trends. You are ready to kick off the project and meet with the customer. And now what should happen? What a plan is? How to find out what needs to be delivered? Where to start?
There is no better way to gather requirements than directly ask your customer. But customer can be a broader audience. In fact, there are plenty of people with different needs and different objectives. It’s necessary to identify those separate groups of users. And at this point, we have reached a first question which should be addressed.
Who is going to use these reports?
The common approach which I encounter is to design One Report for All. In a nutshell, the main idea is to design one report, from one data set and try to bring together all expectations. How does sound for you? Do you think it can be achievable? Do you know what is told about a compromise? That it is not satisfactory for any of the parties. The same result is when we offer one data view for different viewers. Still, we can have the same data set, but, e.g. Marketing Director will look for other information than Controller. Their fields serve for other purposes in the organization; that’s why they need to have tailored alerts and KPIs (key performance indicators). Having in mind those varieties my first question to the customer is about how many different end-user groups we are going to have.
What kind of actions/decisions these people are going to make?
This question is my second crucial one to understand how these groups use and consume data. Understanding of actions and decisions giving me the most valuable information about the true purpose of having reports/dashboards in place. Thanks to that, I can design dashboards with higher adoption, because they respond to daily challenges. Another benefit of this approach is to establish a better and stronger relationship with the customers, who feel that they are heard. What is more, sharing such an attitude we can become a partner, not just the provider.
Of course, those are only opening questions, because just right around the corner, the following questions are waiting: ” What business needs do those groups have? “ “How often they use data?” “Which channels they use to communicate/consume data?” “Which devices they would like to consume data on?” “What is the data literacy level?” “Is the data culture established within the organization” “What are the habits and behaviour patterns regarding data usage?”
To collect answers to these questions, the best way is to hold interviews or workshops with representants of each group to learn their perspective.
Tangible outcomes of KYC.
The success of all projects lies in the preparation. KYC process in BI world is nothing more than a sequence of steps leading to create the vision of the final product. At the end of this stage we should have some tangible outcomes: – well-defined groups of end-users. Those groups are called PERSONAS – fictional characters who represent types of users. – description of product vision: scope, KPIs and alerts list, UX & UI assumptions and foundations.