So, how do we get over the "needle in a haystack" problem? How can you organize data so that you can find it again, without needing to know where it is?

A relational system: the "give me what I want" method

If the problems all stem from having a hierarchical structure, and levels, and branches, and folders, then the solution is, simply, to do away with all of those things. Instead of organizing data by filing things into folders, you organize by tagging things with categories, or attributes. So then the question changes, whether you're organizing the data or retrieving it, from "where do I put it?" / "where is it?" to "what attributes does it have?" / "what attributes do I want?". So, to put it another way, you organize things not by putting them in a particular place, but by tagging them with as many attributes as you like. Then, when you want to retrieve that item, you select things that have the attributes that you want. (And / or that don't have the attributes that you don't want.)

A different structure: goodbye tree - hallo rainbow!

For this new way of handling data management, we need some new computer methods. Two, in fact; a new GUI, the rainbow; and an internal data structure to store attributes and associated markers and pointers to data. The internal workings are a bit technical to go into here, but let's take a look at how trees compare with rainbows. Here is the folder structure from the "problem" page, side-by-side with an equivalent rainbow structure using the same attributes:

OK, so what have we gained with the rainbow method? And have we lost anything? Let's run through some of the features of the two systems:

Pigeonholes vs transparency

With a folder system, you're looking at data one folder at a time. The items in any folder have the attributes of that folder, plus the attributes of all the folders higher-up in the branch. So, if we opened the second folder in the list above, we'd see fonts that were Heavy, and also Serif. On the right, is the rainbow way of looking at the same fonts. We've retrieved the same data by selecting for Heavy, and also for Serif. But the big drawback with the tree structure is the "one folder at a time" nature. You look inside a folder, and you see what's inside that folder, and nothing else. You can't open up three folders at the same time. So, if you wanted to look at all the Serif fonts together, well - you can't. All you can do is to open each of the Serif folder's three sub-folders one at a time, and look first at the Serif + Heavy fonts, then at the Serif + Light fonts, and lastly at the Serif + Outline fonts. But with the rainbow, this is easy; all you need to do is remove the filter on "Heavy". And then you've got all your Serif fonts in one list.

There's more, too. The rainbow doesn't just make it easy to retrieve the data items you want, it makes it easy, with the markers, to see the qualities that the items have. This isn't possible at all with the folder system; you would need to bolt on some other method like maybe icons or text descriptions next to the items. Having all possible attributes together in one place, and a simple, unified way of showing the presence of those attributes, makes interrogating data very straightforward and very quick.

Rigidity vs flexibility

With a folder system, the hierarchy that gets built is a fixed structure; you can't mix-and-match the branches on the fly. You go to a particular place in the structure, and open the folder that's there. Inside it, you might see some data items, and you might see some sub-folders, or you might see nothing. With the rainbow, because there is no structure, and all the attributes have the same value - you don't have "boxes within boxes" so there are no levels and no "depth" - you can filter on any combination of attributes you like. And, you can filter negatively as well. So, you can say "show me all the fonts that aren't Script".

Single-use vs versatility

The structure of folders gives you a system of pigeon-holes to put things away into, and that's about it. If you wanted to add ways to do, say, text search and searchable meta-data, you'd have to put the mechanisms for doing those things somewhere other than in the pane where the folders are - there isn't any way to "superimpose" these on top. With the rainbow, you can use the bars themselves to do both these jobs. There's no need for any extra Windows paraphernalia, so there's no need to take up any screen real-estate beyond what the bars are already using. Here's how FontRainbow does these jobs:

Complexity vs Simplicity

With the folders, we found we ended up with Outline fonts in three different places. Duplication is always a problem with a hierarchy system, because the same attribute will often crop up in different branches. With a rainbow, an attribute only occurs once. That makes life easier both when you're organizing things and when you're retrieving them. When you're organizing things, you just set an attribute on by going to it and clicking. You don't have to worry about any other attribute than the one you're setting. With the folders, if the folder you are dropping into is at the fourth level down, you do have to worry about all the attributes in the three levels above - you need to be hitting the right place in the structure. Similarly, when searching, instead of hunting around for the folder where your item is, you just set as many filters as needed to "zero in" on your item.

Drag-and-drop vs single click

Let's look at how these two methods let you organize data. With folders, you need to put something somewhere. You put a file into a particular folder. Either you drag the file to the folder and drop it, or you select the file, copy it, go to the folder, select the folder, and paste the file in. Either way, it's a laborious process. With the rainbow, you do it with a single click. How is this possible? Take another look at the two GUIs:

Because with the rainbow the attributes and the items intersect each other, we can use the little intersection rectangles in two ways; one obvious, and one not so obvious. The obvious way is that we can display a marker to show that an item possesses that attribute. The not so obvious way is that we can use the rectangle as a control button, to perform the action of toggling the attribute on or off. That in turn means that we need fewer actual control buttons on the screen, so the GUI can be less cluttered. In fact, this "virtual control button" can perform "real" actions as well; in FontRainbow, you install and uninstall fonts by clicking in a bar. It couldn't really get any simpler!

Reservations?

You may well be having reservations about this idea. We've had them ourselves. Here is a list of some which have occurred to us, along with replies to them.

There are only seven bars - I would need a lot more than seven attributes!

There are various ways around this; here are three.

1. Different "pages" of attributes can be selected. In our FontRainbow program, there are 6 pages on the left (for program-defined attributes), and 9 on the right (for user-defined attributes).

2. Rather than numbering the pages, a drop-down can be provided showing a list of attribute categories. (see the Digital Asset Management mock-up in the Variations page). The user can maintain and add to this list, giving an unlimited number of attribute pages.

3. The attribute pages could be made scrollable, with a scrollbar immediately above the main window scrollbar. Then, the limit of 7 attributes to a page would disappear.

I don't like throwing away the structure of folders. Things need to be in their own place!

This is the really big, fundamental, objection to this idea. We've become accustomed to organising data by "putting things in their place", and to finding them by going to the place where they are. The reason for this isn't hard to find; it's how things work in the real world. We file bits of paper away in a folder inside a drawer, and the drawer is one of several drawers in a filing cabinet, which is one of several filing cabinets in an office. That's a hierarchy. It was only natural that the first computerized systems would replicate this; systems like Windows Explorer are a metaphor for a real-world way of doing things.

But computers don't have to be limited to real-world ways of doing things. The relationships window in MS Access isn't describing anything that could be physically built with modelling clay and bits of string - it's a system that can really only exist inside a computer. Similarly with the rainbow. It's a computerized way of organizing real things - files - by using the relationship between those things and attributes which they possess (or don't possess). The difference with the rainbow - the key thing that a database like Access doesn't have - is that the attributes don't have to be part of the data; they can be created at will and "overlaid" onto any set of objects.

Having said that, there is still a valid objection that you need a place for things to reside in. That way, you can expect that when you look in a certain place, the same set of objects will always be there. The rainbow doesn't give you that sense of things being in their place, so that you can always get them all back again. Now, while it's debateable whether you actually need the "sense of place" for all kinds of data, there isn't space to have that debate here! (but see the next reservation). What we can do, is provide some workarounds for setting up subsets of your data items that stay around for later. What the problem boils down to, with the rainbow, is this: having set 5 filters, and zeroed-in on 50 items, how do I "save that search" so that the 50 items can be brought back later? Two ways come to mind; one, a "save search" button could be provided, to let the user name the search and save it away, so that the same filters could be set at a later date; and two, the user could set up a new attribute, with a name formed by combining the names of each of the 5 filters. So later, you just use the one new filter to get back to the 50 items.

Some things do need to be arranged in a hierarchical way.

We accept that there are times when a hierarchy is the way to go. For example, the standard Linnaean system of biological classification is by nature a true hierarchy, and we wouldn't suggest using a rainbow for it. The question for us is really - do things in a computer system generally fall into a natural hierarchy? To which we would say, generally, no.

and finally...

I couldn't bear to sit looking at a rainbow all day long!

The rainbow, with the colour bars at their most intense, can be rather overpowering to look at for any length of time, especially late in the evening when you're tired. Of course, the colours can be made adjustable so that the user can configure them, but the key thing is that what matters is the shape of the bars, rather than the colours. There are lots of other looks that the GUI can have - we show some in our Variations page, here.