Introducing the flowgraph

As I’ve been messing around with a pile of event data for Twitter I’ve been mulling over aspects of visualizing the data in a way that helps to unlock the meaning and insights contained in that data. Along the way I’ve come up with a new way of visualizing data and I’d love your thoughts to help me make it better.

Twitter events, while often interesting in their own right, have things in common with many other types of human experiences. They occur in time in a linear fashion, they contain a continuous flow of “things” (tweets in this case), those things have a variety of quantitative characteristics as well as a variety of qualitative characteristics.

While my initial examples of the flowgraph will be using data from a Twitter event, the flowgraph could just as easily be applied to military incident reports, stock trades, fire station responses, customer complaints, weather events and so on. Basically anything which occurs in a single linear timeframe might be a good candidate for a flowgraph.

I’ll outline some of the features and ideas for making flowgraphs below. Along the way I will flesh out an example showing each feature and how it might be used. Keep in mind that how you might use a flowgraph will depend on your research question and your available data set; my example isn’t the only way to make a flowgraph (I hope!).

Note on the example BloombergTV/Burlington VT data set

My data set isn’t nearly so important as the possibilities listed above, however. It’s simply a bit of social media data gathered via Radian6 covering the Burlington Vermont hashtag (#BTV) from 11:00am to 11:30am on Dec 4, 2012.

This is an hour and a half before BloombergTV was to air (cable?) their exclusive interview with President Obama–his first following the November 2012 election, when the “fiscal cliff” was a significant topic of discussion. Some time earlier in 2012 this BloombergTV’s social team decided to use the #BTV hashtag with the objective “to prompt smart conversations.” (Lynn Monty “Burlington Defends Twitter Turf” Burlington Free Press 12/4/2012) Burlington Vermont Twitter users have been actively using the hashtag since 2007.

The research question I will be exploring is whether the strategy and tactics employed by the Bloomberg TV social media team was effective for prompting smart conversations.

How a flowgraph works

A flowgraph is a map of time and activity for linear events. It’s purpose is to improve decision-making for event-based experiences.

Basic structure of a flowgraph

The core of the flowgraph is a vertical stack of boxes. At either end of the stack is a time. In this way, the timeframe under examination is established.

For example: This is a flowgraph stack that covers the time between 11:00am and 11:30am (Eastern) on December 4, 2012. At this point, the use of the flowgraph doesn’t really help us any. It shows us how many tweets occurred over a timeframe. It would be much simpler to use a sentence: “Between 11am and 11:30am Eastern on Dec 4, 2012 there were 96 tweets to #BTV.”

Event boxes, height and width

Each box represents one of the events being tracked. The box has a height which is keyed to some variable related to either time, space or attention. The width of a box may be keyed to a variable not necessarily related to time or space or not keyed to anything.

For example: In this flowgraph each box represents a tweet that occurs in the timeframe determined above. The height of each box is related to the character length of the tweet represented. Each box is a height between one and six units.

The one through six scale was chosen because of the way people interact with Twitter. In usage, all Tweets flow into the audience’s interface one at a time and stacked according to when the Tweet was posted (newest on top). Some tweets, being longer, can take up to six lines in an interface. Others, being very brief, may only take up one line.

Because of this variability, some tweets will take up more space in the interface. I wanted to reflect this haptic reality in the flowgraph.

We can now see which tweets take up more space in the audience’s display device. More interestingly we can see contrasts between the size of tweets scrolling across the interface. Still not terribly useful though. It is, in effect, a tiny version of the entire stream should you take a screen shot of all the tweets and then paste them together in Photoshop.

The width of each box could be keyed to another variable as well, which I’ll discuss below. But for this example the width exists primarily to make the flowgraph easier to read. Just as a book requires margins so that we may have some place to rest our thumbs, so too this flowgraph gives something for the eyes to grip while reading.

Multiple stacks

Some events have a variety of qualitative differences between them. If the events can be segmented or sorted based on these qualitative differences, it can be useful to put them in separate stacks. In this way, the flow of activity in one channel can be sorted and sifted.

Many such stacks could be used if there were a wide array of qualitative differences being examined. However I suspect there is diminishing returns in this regard.

Using multiple stacks is similar to parallel coordinates‘ method of displaying multi-dimensional information in a single graph. The arrangement of the stacks may or may not be arbitrary and the rearrangement of the stacks may yield different insights.

For example: This flowgraph features two stacks. On the left are tweets which are qualitatively on-topic for Burlington Vermont. On the right are tweets which are qualitatively on-topic for BloombergTV’s interview with President Obama.

Note there is one tweet which is sort of centered across both stacks. This tweet was not on topic for anything (it was spam). I didn’t want to leave a gap because the event was definitely in the stream and I didn’t think it merited it’s own stack. So I just shoved it in the middle. Probably not very elegant, but honest at least.

At this point we can begin to see which conversation is more prominent to someone choosing to view the hashtag stream during the time under examination.

Equalizing time

Each event, regardless of its physical characteristics, also exists in a steady flow of time with other events. In fact, the more events occurring at any one time the less likely a person will be able to comprehend them all.

In order to recognize that time is steady and linear, each minute of event boxes are grouped together and then made a consistent height. A choppier visual result indicates a higher velocity of activity–a decrease in time available for the audience to consume each event.

For example: The flowgraph has now been adjusted so that time flows evenly from 11:00am to 11:30am. Areas of increased activity are now clearly visible. Also, areas of no activity result in gaps in the flowgraph.

With time being represented evenly we can now observe the pulse of the event we are examining.

Basic capabilities of the core set of flowgraph features:

As analysts, the above features might help us: identify areas to dig deeper in a large data set, identify regular rhythms in the event stream, identify regularity and contrast in event size, identify patterns in qualitative differences within the stream.

Some additional flowgraph features

Depending on the richness of our data and our research question, we might have some additional things to add to our flowgraph.

Event sources

Some sort of graphic indicator could be used to indicate originators of events that are of particular interest. While the stacks segment by broad qualitative difference, some other visual indicator would need to be used to highlight specific actors or event origination sources.

For example: This flowgraph has used used orange to color all tweets originating from Bloomberg corporate accounts and Bloomberg employees.

This helps us to understand what actions Bloomberg is taking to further their objective of initiating conversation. It also helps us to identify which tweets are from accounts not owned/controlled directly by Bloomberg TV. This helps us identify the available pool of participants who are eligible to be considered having “smart conversations.” The Bloomberg TV accounts can have smart conversations whenever they like whether or not they use #BTV in their tweets, of course.

Event relationships

Some events are related to one another. Something happens and as a result something else happens later. We might want to use something to indicate a relationship between events.

For example: One cultural practice of twitter users is the “retweet.” The “retweet” (or sometimes simply “RT”) is when one user repeats another user’s post. This is usually done verbatim or with very minimal editing to allow the retweet to conform to Twitter’s 140 character limit.

In this flowgraph I have place a connecting line from each original tweet to any retweets it may have inspired in users later in the timeframe.

With these lines in place we begin to see the nature of discourse in the different stacks. We can see whether they are more broadcast-like or more conversation-like. We can also, since the stacks are divided, see if there is a different kind of discourse happening in each of the two competing topics.

This might be a first step at evaluating the existence of conversation and also identify patterns in the “prompting” of potential conversations. It also reinforces the difference in segmenting between the two stacks. Notice that no lines connect from the Bloomberg stack over to the Burlington stack or vice versa.

Quality of relationship driven content

Not all relationships are of equal quality. Some are very one-sided in which a participant does whatever the other participant does; like children playing follow the leader or aping the movements of an older sibling. Some relationships are more equal, in which new information is added to the stream by all participants.

We might want to indicate the quality of content resulting from an event relationship in our flowgraphs. Perhaps shade would be an appropriate method.

For example: In this flowgraph there are four levels of shading: solid for a tweet consisting of all original content, slightly less than solid for a retweet which contains additional commentary, very light shading for a retweet which contains no original content and no shading/outline only for a tweet that is off-topic for both stacks (i.e. noise to both segments).

Shading at these four levels gives us the ability to see which relationships are generating the most original content as well as looking at each stack on the whole to figure out which of the two topics is generating the greatest amount of original content.

Further dimensions available

There is still some room in the flowgraph for additional information. Symbols could be added to the margins, shapes other than boxes could be used, shading or line style could be used on relationship lines, textured fills could be used on the boxes and so on.

At some point it will become unwieldy of course. But those options are there if it makes sense.

It might make more sense, however, to simply strip away everything except whatever it is you might want to focus on and present multiple flowgraphs to help keep your reporting focused.

For example: One of these flowgraphs uses a dot in the margins of the flowgraph to indicate the first time a particular account tweets during the course of this timeframe.

The other flowgraph strips away all boxes except for the first time a particular account tweets. Note that in this variation an empty space does not necessarily indicate time during which there were no tweets. The flowgraph is giving a visual representation of when people were speaking for the first time in the channel.

In my opinion, the second flowgraph does a better job of letting us know how people began to join this stream chatter in the time preceding Bloomberg Television’s interview with President Obama.

The first, however, may be better at showing how many accounts stuck around and began to interact with others. One of the hallmarks of conversation is that people speak, listen and then speak again and so on.

This is one of those areas where you have to use your judgement and pick what works best for your research question. I try to default to a less-is-more approach because I don’t want to make things too cluttery. But on the other hand, some genuine insights might be gained if you see it all together.

Some questions I’ve been asked so far

Thanks to everyone at the Jan 30 Burlington Web Analytics Wednesday as well as my friends on Twitter for asking some questions and making suggestions. I hope to get more so that the flowgraph way of examining event data can be improved.

Can I make my own?

Of course! I’m not copyrighting the flowgraph or anything. It’s terribly to difficult to copyright something like this even if I wanted to.

My goal in sharing this technique is that hopefully other people will use it. It certainly would make me feel nice if you mentioned me as the format author now and then, but it’s not necessary.

I hope you do use flowgraphs to help understand event data and relationships better. I’d love to see your work and how you use it.

Why not go with a horizontal layout for a flowgraph?

While the horizontal layout will be familiar to those who use the x-axis of a Cartesian grid for time and also to video and audio editors, I prefer the vertical axis.

With more than one stack there will be an inevitable judgement or implication that one is placed up and the other is placed down. We are used to reading line graphs where up and down have intrinsic meanings. But in a horizontal flowgraph the relationships may be arbitrary and not up/down; the stacks have qualitative differences and may have no relationships to one another at all.

Also, the human eye is good at scanning horizon lines and I would prefer to put this to use in examining the differences in behavior between the multiple stacks than in perceiving the differences between the size of the events. But this will be dependent on the research question at hand.

Why don’t you use follower count as the width of your boxes in your example?

This has more to do with the research question of my example: Was Bloomberg TV social strategy and implementation successful at prompting smart conversations?

I don’t believe that follower count is an indicator of whether a conversation is being had or not. If a person with a large following has a conversation with a person who has a small following neither one of them is having more of a conversation than the other. So what value is brought about by  including that in the flowgraph?

There is also a data quality issue here that makes me uncomfortable. A great many followers of all accounts–especially well known brands–are not, in fact, other people. A bot and a human may be equal in the eyes of a social media marketer eager to justify their efforts to an executive who isn’t aware of these issues, but I would prefer to avoid being an accomplice in that sort of thing.

Strategically it’s a losing proposition for the BloombergTV social media team because advertisers want to know that humans who spend money are being attracted to advertising on Bloomberg media properties. By inflating the number of “eyeballs” to include bots they would be decreasing the quality of their audience to advertisers (ratio of audience size to people who take action on advertisers’ messages). Once the bots start spending money then this will be less of an issue.

Finally, not all followers to an account see all tweets all the time. Though it’s hard for me to reconcile, not all of my followers read every tweet I publish. The same is true for everyone. This is the nature of stream experiences. Follower count is the only available metric to approximate this, but the value of each grouping of followers varies so widely as to make the number not very useful.

This is one of those cases where just because a metric is easy to gather using computers doesn’t mean it’s relevant to a business objective or useful for making important decisions.

If the intent is to show the potential reach of the conversations that are occurring, it may not be necessary to go through the effort of creating a flowgraph. The flowgraph is focused on understanding actual activity over time using metrics that have a known value: someone really did retweet this, someone really did post a tweet on this topic.

“Reach” or “broadcast metric” stuff is almost always going to make more sense in a table:

Follower counts of tweets to #BTV from 11am-11:30am on Dec 4, 2012 (data pulled a couple weeks later) Source: Thoughtfaucet
	Bloomberg	Burlington
Followers receiving retweets	179,171	54,101
Followers receiving original content (includes multiple tweets)	730,898	24,807
All followers of accounts that were not BloombergTV-controlled (i.e. new reach)	9,272	NA
Total followers of all tweets	900,173	78,908

This table tells all the story you need for reach: If you remove the twitter followers of Bloomberg-owned accounts–which you will get whether you use expend four characters in your tweet to write #BTV or not–then the reach is dismal even before you take into account the issues outlined above (human vs bot followers, are the humans listening right now?). If you’re able, with a straight face, to tell your boss that this followercount data is evidence of having prompted “smarter conversations” then obviously you don’t need any of the flowgraphs anyway.

All of that said, if you can think of a good use of the width of the event boxes then by all means use them.

Perhaps something related to “resources used” would be worthwhile. Then you would see what the effect of your resource usage is in terms of conversation generation.

What about interactive stuff with rollovers and flyouts of specific tweets and animation and all that?

Yes.

Heatmaps, color and sentiment?

Certainly if it makes sense for your research question. I have issues with sentiment analysis because most of it is, in the words of Master Ken, bullshit. But if you have reliable sentiment analysis coding for your data and your research question involves understanding sentiment then I certainly agree that it would make a lot of sense.

One possibility here might be to examine a stream for a corporation name. Have a stack for corporate owned accounts and a stack for everyone else. Relationship lines for corporate responses to everyone else. Color coding for sentiment analysis. Perhaps a flowgraph like this would help to identify whether corporate account activity is successful at changing sentiment of the entire stream. Perhaps it would identify patterns of emergent and lingering sentiment.

More space between events so the distinctions between them don’t get lost?

Yes. I definitely need to spend a little more time polishing this up. I was so eager to get this out there that I didn’t want to go back and re-jigger the spacing. But you’re absolutely right. I didn’t make all this tiny to be obnoxious, I was just in a hurry.

Wait, you did this by hand?

Yes I did. Part of the reason for this is that I wanted to get a serious first hand look at the data. I knew that simply accepting the easy metrics that software services pump out wouldn’t get me anywhere near the nuance required to understand “smart conversations.”

So I manually coded and dealt with the data. Then I built the graphic in Adobe Illustrator, printed it out and presented it at my local Web Analytics Wednesday.

This manual work with the data is what inspired me to develop the flowgraph so I consider it time well spent. In the future I will automate much of this, however. Or maybe you will automate it first and send me a link to the tool that does it. That’d be even better.

Will your example data set be available for download?

Yes absolutely. It’s a fantastic little collection of data for anyone interested in media ecology, hashtags, conflict or twitter events. I hope it will serve as a nice test set for applications related to flowgraphs or whatever else you have in mind.

But it isn’t ready yet. I will release the full 24 hours of December 4th 2012 when I am finished with the analysis of the Bloomberg TV social media strategy which is begun in the examples on this page. It will probably be a simple CSV dump. But I’m open to suggestion on this.

Thanks to:

There are a lot of people to thank in the thinking and doing resulting in this graph format. As with all things like this, the stupid stuff is me and the smart stuff is them. Brett Chalupa was the first person with whom I talked about flowgraphs; he’s the kind of guy you can trust to listen to you talk about a half baked idea. Alan Wolk encouraged me to get firehose data so there wouldn’t be any question about the meaning of whatever results I gather (and I’ll be doing the full 24 hours of Dec 4, 2012–this half hour is just the intro),  Web Analytics Wednesday Jan 30 edition including Jonathan Butler, Steve McIntyre, Benjamin Bloom, Tyler Machado, Amanda Levinson, The Rambling Stump, Dr. Elaine Young and her class of analytics junkies and people I’m sure I forgot. Justin Cutroni gave very good feedback and use case ideas in a very short amount of time. The Burlington Vermont community of Twitterati, the Bloomberg Television social media team including Kirsten Salyer who has yet to call me back and the Bloomberg Television reporters and anchors, especially Matt Miller, who have exhibited a great sense of humor about all of this.