how to manage thousands of virtual machines
Overview - what is Bot Farm
In a Robotic Process Automation eco-system, if each bot is equivalent to an employee who performs certain tasks, then any company can have as many employees as they need at a very low cost if they "hire" bots instead. However, in the real world, each employee has to have a device to do their task, such as download data from a website or reconcile an account. A bot needs a device to run tasks too. If a company hires 100 employees, it probably needs to buy 100 computers. But what if a company developed 100,000 bots that only run one or two programs and simply fulfill a routine job? Physical devices are obviously too expensive and not scalable. Thus we introduced Bot Farm, the virtual machine management platform, to monitor and to command these virtual "employees".
One vs. All
There are three challenges in designing a dashboard for Bot Farm:
How to show the overall performance and thousands of virtual machines on one page?
How to show the information in a meaningful way, instead of just list and squeeze data into a page?
How to help user diagnoses issues quickly from the data we provided?
The dashboard is separated into four sections:
The top left corner usually is the first spot that people start reading, so I leave it for the overall performance to give users a general sense.
The reset 3 parts have strong relationships. The image filter can help users pick the ones that they are interested in, and also determine which VMs will be shown on its right. And the above graph shows its historical performance.
The overall Bot Farm performance gives users a gist of how their farm looks. However, they would also curious about the performance of each type of image, just like a farmer would love to know the health of each kind of crop. However, there are too many types of images, if we show all of them at once, then the image will be super crowded, and thus nothing can be really seen or compared. Thus I utilize the image filter to help users select the ones that they particularly interested in. And using color codes to build up the connection.
At the same time, the image filter can also be used to determine which virtual machines to display in the list. If there are hundreds of images, then we can expect thousands of virtual machines in a Bot Farm. Again, oversharing too much information at a time is as bad as not showing sufficient information. So only show virtual machines under selected images which help users to identify problems, see progress much effectively and efficiently. Thus, the three widgets, the graph, the filter, and the VM list work coherently as a team.
Additional design details
The secret of designing a meaningful dashboard is always thinking about how the information will be perceived. More often, the problem of a dashboard is not lacking information or ugly UI, but unable to lock down target information among all the irrelevant distractions. And the challenge is, dashboard usually used by all kinds of users, and their emphasis is different. We need to provide comprehensive information to suit most use cases but also balance out how they affecting each other. In this process I learned some lessons:
The fundamental issues in most design challenges are to determine what is "Just about right". Just about the right timing to show; just about the right frequency to show; and just about the right amount to show.
In a complex system, sometimes it is inevitable to have too much important information must be showed at the same time. Then the responsibility of guiding users to make logical connections between different modules and making sense from reading the abundant information lays on designers.
A dashboard must be both useful and usable. Fine details like how users navigate between data, access to the different portions of data, picking special ranges, and hiding irrelevant information, etc. are as important as the data we provide.