Any talk about water won't rival the feeling of this swimmer who just jumped in (picture by Horia Varlan on Flickr)

Or: Why talking about an experience is no substitute for the experience.

This week I led students of Latin America Studies at Georgetown University through a Net-Map exercise (Thanks to their teacher Patricia Biermayr-Jenzano for organizing this!). They chose their own questions (a wide range, from personal family disputes to crime reduction in a Latin American small town) and started mapping it after a brief introduction. All of them had read some of my papers and case studies before, so one of the things that struck me in their feedback was how different Net-Map looked to them when they read about it and when they actually did it. Some of their comments:

“I initially was skeptical because I did not understand why a simple activity could be a method for creating social change.  Net-Mapping allowed me to view the world differently.  Granted, stepping back and analyzing the degree of influences in our lives should be a natural process, but it is something that we do not do visually.  By doing this activity and visually seeing our influences, it breaks the ice and fosters dialogue in a non-confrontational way.” 

“The level of sophistication of the tool far exceeded my personal expectations.  I was skeptical not because of the materials involved in the process (paper and pen) but because of the difficulty in determining who influences whom in most of the research in which I have participated.  I think the greatest advantage of the Net-Map system is the ability to look at an activity from a variety of levels.  My group worked on the scale of the individual, but seeing the work of the other groups made it obvious that Net-Map can be transferred to an organizational level or even perhaps to an international level.” 

“I had never done net-mapping or anything alike before. Honestly, when listening to the explanation I thought it was kind of a game. However, after doing the exercise I actually realized the great value it has. Using this hands-on method of visualizing problems or activities I believe is really useful. I believe that great ideas and problem visualization can be seen that may not be realized using other strategic methods.”

Yes, I fully realize the irony of this post, because, as I said in the introduction: talking about an experience is very different from experiencing it. So, get some pens, post-it notes and toys, print out the instructions, come up with a question that bothers you and involves many different actors and see what happens if you try mapping it. You might not start out as an Olympic swimmer but rather splash around in the shallow pool for a while. But even that will be a more interesting experience than reading stories about water, wouldn’t it?

This is not the kind of Agile I'm talking about (picture by Double--M on flickr)

Last week I worked with an Agile coach who helps large organizations to move their software development from traditional waterfall programming to adopting agile approaches. What does that mean? Well, waterfall programming means you start out by telling the programmers what you need, then they go and program for a few (or more than a few) months and finally come back with a program for you to use. Now you can see whether you actually knew what you wanted in the beginning and whether the finished product fits. In an agile approach the programming cycles are shortened to weeks and at the end of each iteration stands a good enough product that you can start using and trying out, giving feedback to the programmers so that they can go back to tweak, adjust and make it fit.

One of the great and scary things about becoming agile is that it doesn’t just mean using a different kind of product in the end. But that it means significantly changing processes, power and incentives within the organization. So introducing agile is not just a technical switch but actually an organizational change effort. And this is why my colleague proposed that we Net-Map it.

So at the beginning of a 1 1/2 year project he has just started we met with the three leading managers who oversee the agile implementation for this international corporation. And asked them the simple and difficult question: Who are all the actors who will influence the success of the project (positively or negatively)?

What did we find out? Well, my colleague now has a list of people he wants to invite to the first planning round. And within this list, he knows of a few people who need special attention, e.g.:

• The social integrator, that everyone feels comfortable going to with new ideas or the need for feedback.
• Some actors from neighboring domains who fear that their influence might be diminished by the implementation of agile.
• The strongest driver of the process in the leadership team.

He has more clarity about the drivers that motivate the different people involved and their priorities, especially when it comes to the question: “Is it more important to get stuff done and show results fast or to implement and document processes that others can follow in the future?”

Also, mapping out the whole situation provided a great opportunity to dig deeper into the history of this project, the divisions and people involved and how their past experience with each other might influence their ability and willingness to work together on this project. This specifically is an area where external consultants can easily step on landmines from conflicts they didn’t even know existed…

And finally, working with the project leaders on this and giving them the space to draw a map of their views and experiences, allowing for disagreement and exploration as well as finding a shared core, was a great way of laying the ground work for a longer process of collaboration, getting to know each other better, seeing what their priorities and worries are and reassuring them that we have heard.

This is the collection of keyword pairs that appeared in two clusters of people who Tweeted about “Paul Ryan”, the Republican Congressman from Wisconsin who delivered the GOP rebuttal to the 2011 United States State of the Union Address.  This network illustrates the ways that certain word pairs appears only or predominantly in one cluster (colored here Red and Blue) or the other. Terms that appeared in both clusters appear as purple.

Social networks are built from relationships between people.  Keyword networks are built from relationships between words and other text strings.  When two words appear in the same message, sentence, or alongside one another ties of different strengths are created.  The networks that result can illuminate the relationships among topics of importance in a collection of messages.

Markus Strohmaier from the Technical University Graz (TUG) along with Claudia Wagner gave us inspiration in a paper:

C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)

in which they defined a range of ways two words (technically these are strings, they may not really be words) can be associated with one another.  Words could be linked if they are in the same tweet, next to one another, or sequential among other ways to link terms.

NodeXL has not had any features for exploring the networks in texts.  Now with the addition of a new macro from Scott Golder, it is fairly simple to extract pairs of keywords from collection of tweets.  NodeXL’s Twitter importer can optionally include the content of the tweet that included the search term and this column of text can now be processed itself into a new network based on the ways words appear together in tweets.

This feature builds on the work of several people.  Scott Golder from Cornell started the ball rolling with a simple but effective VBA script that allowed others to build and refine the models of what counts as a tie between two words.  Vladimir Barash added several refinements including support for stop word lists to remove common terms.  Scott then picked up the code again and added a set of features for selecting the nature of the graph and making it easier to select the options needed.

The code for the Keyword Network macro is below.

The instructions to use it take a few steps to complete:

1. Create a new workbook, eg a list of tweets or an import from a Twitter search, whatever. Save it as .xlsm. The m is important. This can be an existing NodeXL workbook.

2. Go to Developer -> Macros. Make up a name; it doesn’t matter because it’ll get overwritten. Then press Create. the VBA window will open.

3. In the big text are that says “Sub whatever() End Sub”, select all that text and delete it. Paste in the contents of the text file below.

4. Go to Tools->Reference. Check the checkboxes for “Microsoft Scripting Runtime” and “Microsott VBScript Regular Expressions”. Press OK. Save the file (File->Save) then exit (“Close and return to Microsoft Excel”).

5. Now go to Developer -> Macros. Choose CreateWordNet and press the Run button.

6. It’ll ask you for a worksheet name, a column and a start-row. Then it’ll create a new worksheet with the edgelist in it.

The edge list is not directed (there isn’t really a concept of direction in “co-occurs”) but is weighted. Each pair is weighted by the number of times it appears.

This version also includes options for edge creation.

First, it is now possible to suppress edges of weight=1, which is helpful in getting rid of a lot of garbage.

Second, it is now possible to defined edges by adjacency or co-tweeting. Given a tweet of words “w1 w2 w3″ adjacency will give edges w1-w2 and w2-w3, while co-tweeting will give edges w1-w2, w1-w3, w2-w3.

For edges defined by adjacency, you may choose directed or undirected edges. So a tweet of “Marc Smith Marc” (for example) would generate the weighted directed edges Marc,Smith,1 and Smith,Marc,1 while the sole undirected edge would be Marc,Smith,2. That is, for undirected edges (where ordering doesn’t matter) the words are alphabetized.

An illustrated guide:

Start with a NodeXL workbook with a column of text for either Vertices or Edges (or any column of text).  Here we have the tweet text of a recent Twitter Search Term network query.

Select “Developer” from the Excel menu and create a new Macro.  I take the text of Scott’s macro and paste it here, replacing everything else in the code buffer.

Note the selection of Tools>References> needed to run this macro!  Select Microsoft Scripting Runtime and Microsoft VBScript Regular Expressions 5.5.

Running the Macro:

Scott’s macro presents a series of dialogs to the user (I believe we could do this in a single dialog when we revise):

First we specify the worksheet in the workbook containing the text column to process:

Next we specify the column containing the text to process:

Next we specify the row in which the text starts in that column:

The macro will copy an edge attribute forward if specified (note, I think the *last* attribute for any AB pair is what is reported).

The user is asked if the results should omit the singleton edges, which can be useful.

Edges can be defined as co-sequential or co-cell: ie. ABCD can generate AB, BC, CD or AB, AC, AD, etc.

Users select if they want the edges to include their reciprocal (i.e. generate a “BA” edge for each “AB” edge).

The result is a worksheet with word pair edges and the weights of their frequency of occurrence.

This worksheet can then be imported into a separate NodeXL template using the Import from Open Workbook feature:

This generates a keyword network that looks like this:

We will be working on a revised and updated version of this workflow in the coming months.  For example, this is a possible UI revision:

Create Word Network VB Macro Vdb5//

 

Interested in applying social network methods to better understand the structure of your business or organization?

In collaboration with Optimice, I will teach a workshop on Social Network Analysis for enterprises, organizations, and businesses using NodeXL.

• Self-paced e-learning (4 hours)
  • Introduction to Social/Organisational Network Analysis
  • Network patterns and metrics
  • Software tools for network analysis
  • Managing an ONA Project
    
• Module 1: Scoping your ONA Project (2 hour virtual session hosted by Patti Anklam)
  • Determining which business problem to solve with ONA
  • Review of case-studies
  • Determining your questions
• Module 2: Setting up your ONA survey (2 hour virtual session hosted by Cai Kjaer / Laurence Lock Lee)
  • Setting up your survey
  • Working with mailing lists and other lists
  • Creating relationship sets and network questions
  • Previewing and launching the survey
  • Tracking progress and downloading responses
• Module 3: Visualise networks with NodeXL (2 hour virtual session hosted by Marc Smith)
  • Getting started with NodeXL
  • Calculating and visualizing network metrics
  • Preparing data and filtering
  • Importing data from Social Media tools
  • Clustering and grouping

A number of ONA Practitioner Courses are available to suit the timezones of participants located in the US, Europe and/or Asia-Pacific (but not restricted to these regions):

Course Code Date and Time Time Zone Payment OPC-2012-9-EUR 29 February 2012 to 27 March 2012
(Registration deadline is 15 February 2012)Module 1: 13 March 2012 (10am – 12pm)
Module 2: 20 March 2012 (10am -12pm)
Module 3: 28 March 2012 (3 – 5pm)Self-paced to be completed before starting module 1. Europe – London GMT $US 1,599
OPC-2012-13-APAC 27 March 2012 to 25 April 2012
(Registration deadline is 13 March 2012)Module 1: 11 April 2012 (11am – 1pm)
Module 2: 18 April 2012 (11am – 1pm)
Module 3: 25 April 2012 (11am – 1pm)Self-paced to be completed before starting module 1. Asia-Pacific – Sydney EST $US 1,599
OPC-2012-17-US 25 April 2012 to 22 May 2012
(Registration deadline is 11 April 2012)Module 1: 8 May 2012 (4 – 6pm)
Module 2: 15 May 2012 (4 – 6pm)
Module 3: 22 May 2012 (4 – 6pm)Self-paced to be completed before starting module 1. Americas – New York EST $US 1,599

As many of you know, this week thousands of people mobilized to protest two laws being considered in Congress: the Stop Online Piracy Act (SOPA) and it’s Senate version the PROTECT IP Act (PIPA). Several Internet mainstays, such as Wikipedia, Reddit andy O’Reilly blacked out their sites to protest the bill. For some information on why this legislation is so dangerous check out this excellent video by The Guardian.

The mobilization against SOPA/PIPA also included many grassroots efforts to contact Congress and demand the bill be stopped. Given the attention the bill was getting, I was curious if there was any surge in discussion of the bill by members of Congress on Twitter.

So, I created a visualization that is a cumulative timeline of tweets by members of the U.S. Congress for “SOPA” or “PIPA.” To see if there was any surge, check out the visualization for yourself.

People (boxes) who tweet and core words (bubbles) they use

The fact that today’s social movements, from Occupy Wall Street to the Arab Spring, rely so heavily on twitter and similar communication tools, pose an amazing chance for researchers and other curious people who want to understand who moves these movements. The other day I discussed with a friend what kind of networks you want to look at to better understand this and I’d propose three different kinds: People networks, semantic networks and two-mode people/semantic networks.

People networks are the easy intuitive ones: Who follows whom? Who re-tweets whom? Looking at this will help you understand who the leaders, boundary spanners, broad-casters are.  Most likely, for an issue that manages the step from tweet to action successfully, you will look at a core-periphery structure, with a small inter-connected core (who might also communicate regularly outside of twitter) and a large periphery of followers, who are less inter-connected but look at the core for calls to action and thought leadership. Over time, different clusters might pop up as their own sub-cores or even take over from those initially starting the debate.

Semantic networks look at which words appear together in the same document (a document could be a single tweet, a string, all tweets from one person, whichever works). This can tell you something about the discourse around your issue: Is it just one large well connected issue or are there different schools of thought (more moderate and more radical for example or more philosophical versus more pragmatic and logistics oriented)? You might see that things evolve over time, for example it might be that the movement starts out united behind one cause (“Let’s overthrow the government!”) and after that is achieved, the debate disintegrates in many different camps (moderate and radical islamists, market oriented democrats, socialists etc.).

And to really understand how this development of the debate and the connections between the tweeters hang together, you want to look at two-mode networks. But I have to warn you, they are the least intuitive. In a two mode-network you look at two different categories of things, for example people and words and how they connect to each other. So, there are no direct links within one category (no people-to-people links or word-to-word links). This picture shows you: Who uses which words? Who is connected by being part of the same discourse (even if they have no direct link to each other)?

By looking at all three of these together, you can see who the leaders are, what their role (content) in the movement is and how that develops over time. And if you can compare either different incidents or different points in time, you will learn something about the network structures that are best suited to lead from tweet to action.

Last night Mike Dewar presented a wonderful talk to the New York Open Statistical Programming Meetup titled, “First steps in data visualisation using d3.js.” Mike took the audience through an excellent review of d3.js fundamentals, as well as showed off some of the features of working with Chrome Web Developer Tools. This is one of the best talks we have ever had, and if you have had any interest in exploring d3.js, but were intimidated by the design concepts or syntax, this is exactly the talk for you.

Also, Mike’s slides were all designed using d3.js and are available for download on his Github account: https://github.com/mikedewar/d3talk.

Many months ago I blogged about the research that John Myles White and I are conducting on using Twitter data to estimate an individual’s political ideology. As I mentioned then, we are using the Twitter activity of members of the U.S. Congress to build a training data set for our model. A large part of the effort for this project has gone into designing a system to systematically collect the Twitter data on the members of the U.S. Congress.

Today I am pleased to announce that we have worked out most of the bugs, and now have a reliable data set upon which to build. Better still, we are ready to share. Unlike our old system, the data now lives on a live CouchDB database, and can be queried for specific research tasks. We have combined all of the data available from Twitter’s search API with the information on each member from Sunlight Foundation’s Congressional API.

To show the power of the database, I decided to use my newly acquired d3.js (sick of it yet?) skills to put together a tool that allows you to compare the monthly Twitter activity of all members of the U.S. Congress on Twitter for 2011.

Simply choose a politician from the drop-down menu (alphabetical by surname) and the graph will update with their activity data. If you want to reset the graph, just click the “Clear selections” button.

Feel free to add as many members as you like, but the dimensions of the visualization max out around 9. I have been playing around with for awhile, it’s fun! Oh, and if you choose a member and nothing happens it is most likely because that person didn’t tweet anything in 2011. I could have built-in error-catching or some warning. Also, to clear things you need to re-load the page. I’ll leave real UX to the professional web designers.

Back to the data. Unfortunately, the database is sitting on a server that cannot process many requests (read, web-scale) at a time. In fact, this blog post may bring it down! As such, if you are interested in getting access to the database please contact me directly. But be forewarned, working with this system and CouchDB requires a mature understanding of several tools and languages; including but not restricted to; curl, map/reduce, Javascript, and JSON. And that’s before you have even done any analysis.

Many people have asked me about working with Congressional Twitter data, so I hope this data can be useful. Please feel free to reach out if you have any questions.

Beyond the org. chart: Conflict and personal friendships influencing innovation

“It’s time to re-invent management. You can help!”

That’s how the Management Innovation Challenge is introduced on their website, and I though: “Well, if you think so, I’ll help…” So together with my colleague Michael Lennon I contributed a Hack that describes how you can use Net-Map as an easy and approachable tool to discover hidden influencers.  How do you teach people on all levels of an organization how to effectively navigate the “people aspect” of achieving your goals?

If you are a regular reader (or even fan???) of this blog, you know what I’m talking about. If not, it’s a rather brief read. But whether new to Net-Map or experienced Net-Mapper yourself, head over to

http://www.managementexchange.com/hack-129

Look at what we have to say and give us some love by rating our hack and commenting on it.

Oh, and beyond this shameless self-promotion I’d also recommend you go there and read what everybody has to say. Some amazing contributions, all bundled under such inspiring moon shots as:

• Humanize the language of business
• Capture the advantage of diversity
• Make direction setting bottom-up and outside-in
• Build natural, flexible hierarchies.

I will speak this March 4th at the 2012 Predictive Analytics World in San Francisco about ” Crowd Photography for Social Media“.

http://www.predictiveanalyticsworld.com/sanfrancisco/2012/speakers.php
http://www.predictiveanalyticsworld.com/sanfrancisco/2012/agenda.php#

Monday @ 5:25-5:45pm

Track 1:
Social Data Case Study:
Social Media Research Foundation
Crowd Photography for Social Media

Crowds of people gather in social media around many products, services, businesses, and events but they can be difficult to see and understand. With new free and open tools, it is now possible to map and measure social media spaces, capturing the sub-groups and key people within and between them. Learn how to capture social media data and quickly generate a visual map of the crowd. With maps in hand, we will discuss ways they guide a journey to the key influencers and concepts in the crowd.

Speaker: Marc Smith, Director, Social Media Research Foundation

As the United States gears up for what appears to be a long and grueling 2012 presidential campaign, China will also undergo its decennial turnover in presidential power in 2012. Unlike the United States, however, this shift will not involve any campaigning or voting—at least not with the people of China. Instead, this shift is one that is formalized within he Chinese Communist party; but that doesn’t mean that there won’t be interesting shifts and reallocations of power.

This leads naturally to many questions; perhaps most importantly that of this post’s title: Who are the most central members of the China’s leadership as we enter 2012?

Recently, I had the opportunity to work with Recorded Future, a startup out of Boston that specializes in longitudinal entity extraction from the massive amount of open-source data generated daily. For example, they have used their data to predict future patent issues for Apple based on issues raised by their competitors. This analysis includes many entities: Apple, HTC, Samsung, etc.; as well as the patents and law suits.

For our analysis we focused on the China’s leadership, as defined by the CIA World Factbook, and extracted all of named entities in their data for 2011 (over 4 billion events) for which any of the 33 official Chinese leaders appear. The result is a dataset with over 150,000 entities; including people, organizations, and places. To answer our questions, however, I used the co-occurrence of these entities in sentence fragments to build a large network of these entities.

Here I define an edge between two entities as the co-occurrence of two entities in a sentence fragment, which is provided by Recorded Future. Then, by extracting only the entities that are defined as people in the data, I generated a graph with 5,435 nodes and 34,413 edges. Big, but not unreasonable for analysis. Next, I computed some basic network statistics on that graph. As I have mentioned many times before, these measures are often most interesting if compared together. To highlight key actors, I generated a scatter plot of two metrics: Eigenvector centrality and betweenness centrality.

Eigenvector centrality measures the overall centrality of person in the network. It accounts for not only the number of connections a person has, but also the number of connections that person has to others with many connections. People with high Eigenvector centrality will be the most prominent and well-connected actors in a network. Alternatively, betweenness centrality measures that number of paths that go through an actor as a function of the total number of paths in the entire network. People with high betweenness will be those that act as critical bridges or cut-points between two densely connected parts of a network.

When we compare these metrics, as I have above, we can easily identify key actors as those that do not follow the relatively linear relationship between to two measures. Those with high betweenness but relatively low Eigenvector are central bridges within the network. What makes this comparison important, however, is that these bridges do it with few connections—hence the lower EIgenvector. Likewise, those with relatively high Eigenvector but low betweenness are network insiders. They sit inside some central region of the network, but have very few connections outside that region. To further highlight these key actors, I have shaded the data points based on how much they diverge from this linear trend. The network bridges are dark the red, while insiders are dark blue.

The above plot was designed using d3.js and is interactive. When you roll over the data points the “Leader Information” section is populated and identifies who in the Chinese leadership the point represents. If you click on the point, or the photo of the leader, you are brought to their full biography page provided by China Vitae.

What I love most about visualizing data in this way is it leads to many questions. What I love more about creating interactive graphics is it allows for that first layer of questions to be immediately answered, which in turn leads to an even richer investigation.

Many things jump out of the above plot right away:

• We see the obvious placement of Hu Jintao and Wen Jiabao in the upper-right of the graph. This is useful because it confirms that nothing odd is happening in our data: the most powerful men in China on paper are also the most central in our graph.
• Popular press is reporting that current Vice-President Xi Jining will lead the transitional government, so it is interesting to see him clustered closely with Xie Xuren and Zhao Xiaochuan. What will their role be in the new government?
• Why is Yin Weimin, a man with an ostensibly minor role in government, such an important bridge in the network?
• Liang Guangile, the Chinese Minister of Defense, is a key insider. This seems makes sense given the prominence of the military in the Chinese government, but why is he isolated from the rest of the network?

What’s more important than these questions, however, are the non-obvious ones this plot raises. What I need is help from those with a better understanding of Chinese politics. Does the placement of some of China’s leaders seem way off, or does this plot essentially reinforce well-held beliefs about the balance of power? I am very interested in getting other people’s perspectives

Finally, I want to give a special thanks to Christopher Ahlberg, CEO of Record Future, for working with me to on this data and allowing me to publish these findings.

On March 1st I will speak at the 2012 Strata Conference in Santa Clara, California about:

Mapping social media networks (with no coding) using NodeXL

Time: 16:50 on 01 Mar 2012.

Session type: 40 minute presentation

Topics: Visualization & Interface

Description: Maps of the complex connections that form when people link, like, reply, rate, review, favorite, friend, follow, edit, and mention one another can reveal important trends. It is possible to create network maps with free and open tools that identify key people and sub-groups in any social media population with just a few key clicks. Can you make a pie chart? You can now make a network chart.

Abstract: Networks are a data structure common found across all social media services that allow populations to author collections of connections. The Social Media Research Foundation’s (http://www.smrfoundation.org) free and open NodeXL project (http://nodexl.codeplex.com) makes analysis of social media networks accessible to most users of the Excel spreadsheet application. With NodeXL, Networks become as easy to create as pie charts. Applying the tool to a range of social media networks has already revealed the variations present in online social spaces. A review of the tool and images of Twitter, flickr, YouTube, and email networks will be presented.

We now live in a sea of tweets, posts, blogs, and updates coming from a significant fraction of the people in the connected world. Our personal and professional relationships are now made up as much of texts, emails, phone calls, photos, videos, documents, slides, and game play as by face-to-face interactions. Social media can be a bewildering stream of comments, a daunting fire hose of content. With better tools and a few key concepts from the social sciences, the social media swarm of favorites, comments, tags, likes, ratings, and links can be brought into clearer focus to reveal key people, topics and sub-communities. As more social interactions move through machine-readable data sets new insights and illustrations of human relationships and organizations become possible. But new forms of data require new tools to collect, analyze, and communicate insights.

I will present a talk at Personal Digital Archiving 2012 titled “Arc-chiving: saving social links for study“.

The conference will be held on Thursday-Friday, February 23-24, 2012 at the Internet Archive in San Francisco.

News and updates on the conference will be posted at the conference web site, http://personalarchiving.com.

My talk this year will focus on collecting and analyzing connections between digital objects (like users) and the insights these tools make possible.

Abstract: While digital content is archived in various ways, the “arcs” or links among people and their digital objects are not systematically saved. Efforts to store social media often overlooks including data about collections of connections. The Social Media Research Foundation is dedicated to open tools, open data, and open scholarship related to social media. It is producing tools that can collect, analyze and upload social media data, including the arcs that link people and objects. Using the free and open NodeXL application, users can collect, analyze and visualize complex networks and then upload the data to a growing archive on the web at NodeXLGraphGallery.org. As the group of researchers grows, an archive is being assembled to provide researchers around the world with the data about social media needed to understand the ways computer mediated communication tools shape society.

My talk at the 2011 Personal Digital Archiving conference is available through the Internet Archive’s video service:

I will speak and lead a workshop on social media network analysis at Syracuse University on the 19th and 20th of January, 2012.

Ines Mergel is my host.  Prof. Mergel is Assistant Professor of Public Administration, Department of Public Administration and International Affairs, and a Senior Research Associate at the Center for Technology and Information Policy at the Maxwell School of Citizenship and Public Affairs, Syracuse University, NY.

I will speak about the patterns we are finding in the data collected and analyzed by NodeXL.

First, from looking at the date of my last substantive post I owe everyone an apology. I have essentially let Zero Intelligence Agents wither on the vie, and that is terrible. Not so much because I think people are desperate to read it, but because I am desperate to get feedback from people on my projects and ideas.

One such project I have been working on recently is looking at the newly released data on Federal Reserve borrowing of 407 banks and companies during the 2007-2009 financial crisis. I have been looking for data sets to tell stories with because one of the tools I am eager to learn in 2012 is Michael Bostock’s d3.js, a Javascript library for data-driven design (d3, get it?). It is an incredibly powerful tool, albeit very verbose and cumbersome for a total Javascript newbie such as myself

I decided to teach myself some d3 through this Federal Reserver data, and came up with this visualization in the labs section of drewconway.com. The image below is just a snapshot of the visualization, please click through to see the full interactive chart.

Because the data contained so many companies, I decided to focus on only those that were the most aggressive borrowers during the crisis. I defined this as an institution that borrowed more than 500% of its market capitalization in a single day, i.e., 5x its value. This left me with 16 companies. I then also excluded Lehman Brothers, because on a single day it borrowed over 40x its value, which was too extreme an outlier for this visualization.

What’s left are 15 companies that tell a fascinating tale of the turmoil in the financial markets from 2007-2009. What struck me the most about the visualization was how many foreign banks were among the most aggressive. From the snapshot above, you can see that Dexia SA has the largest spike in its trend line. I must admit, I had never heard of this bank, but as it turns out it is a Belgian-French bank, which also happens to also be currently under investigation by the EU.

I would love to get feedback, both in terms of the data as well as the design of the visualization. If anyone has some insight as to what was going on with these banks during this time that might explain their trends, please let me know. Also, as I am very new to d3, if you have ideas on how to make the visualization better I welcome those as well.

UPDATE II: Updated the visualization with a legend and company filter, as suggested by many. I think it is better.

UPDATE: Thanks to @deepfoo for pointing me to this backgrounder from Bloomberg.

A few days ago I was on the phone with a colleague who did a series of Net-Maps with groups of African farmers, asking them where they get their information about improving their farming practice. When we talked about the data she collected, we realized that what her farmers had mapped was like the pipe system (hopefully fresh water and not sewage…): What are all the potential connections that these farmers could use?  That’s an interesting questions. And as the mapping was done with groups of farmers, I am sure that a lot of them learned about information sources they were not aware of before and that drawing the maps together might have helped them to access more and more diverse information afterward. What they didn’t map though was where does the information actually flow; and who provides more fresh water (good, correct, new information) as compared to sewage (old, wrong, useless information) – though some of this information was shared in the discussion.

I’m not writing about this, because there is a right and a wrong approach to mapping out information networks. I think it is important to know about the (potential) connections as well as the flow. And depending on your underlying question and motivations, one might be more crucial than the other. But what is important is to be aware of what you are mapping, just like my friend was, otherwise it is so easy to misinterpret the answers and make up very bleak or overly optimistic stories about the connections that people  have access to or actually use.

Call for Papers: Arts, Humanities, and Complex Networks -- 3rd Leonardo satellite symposium at NetSci2012 Tuesday, June 19, 2012 at Northwestern University in Evanston/IL, near Chicago/IL on the shores of Lake Michigan. Abstract: We are pleased to announce the third... Sune Lehmann http://sunelehmann.com/

My PhD research in Zambia is an evaluation of an NGO program that aims in part to align and coordinate certain activities within the Ministries of Agriculture and Health for improved nutrition outcomes (both food and health being essential elements of good nutritional status, of course!). A key piece of information, then, is how are different players in these sectors interacting right now, and how does that interaction change over the course of the project? Enter NetMap.

The key to the alignment strategy being used in this project is to start at District rather than National level, to create a model of coordination that can be used to advocate for scaling up to other areas or even other countries. Ministry staffing is minimal at District level, so I aimed to interview everybody employed in each District Ministry, from the Directors down to technical officers (around 5 people per ministry), and to snowball out from there to anyone else who came up in the interviews as crucial to the process.

This being the first time I had used NetMap, I was unsure how it would be received- how would people react to being asked to give up an hour or more of their day to draw pictures with an outsider? In anticipation of rejection, I made sure the process looked as professional as possible- putting together a regulation NetMap kit, sending formal letters of invitation to interviews, hiring a highly professional local assistant, and dressing as smartly as I possibly could in sweltering pre-rains temperatures. But the method held true, and just following the steps from actors to links to influence engaged everyone from the moment we started- as I had been promised it would!

Being on a smaller scale than much national-level research I have seen that uses social network analysis, I had wondered if I could use NetMap at the individual level; that is, could I map not only the formal interactions but also the informal interactions between individual players within each Ministry, since it is very likely that personal relationships shape collaboration, particularly in such a small population as in the district capital (a small, one-road town). One of my pre-defined links therefore was informal interactions, and my questions attempted to probe whether person X might have family ties to person Y, or whether person A drinks in the evenings with person B. But it turned out in pre-test that even small-town rural Zambia had too many players in this field for everyone to know everyone; people knew which organizations were doing what with nutrition, but not who was doing it, and the method defaulted pretty quickly back to looking at organizations rather than individuals. Still a very interesting picture, but I wonder if there might be something in this for my future research…

So, now I have a collection of beautifully colorful maps to process and a good idea of local views on the alignment of sectors for nutrition in rural Zambia, so watch this space…

Doodledidoo... (copyright by lourdieee on flickr)

Are you like me, when you try to explain something complicated (or exciting) to others, you quickly grab pen and paper and draw some weird picture or graph that makes absolute sense to you, helps you structure your thoughts and maybe (or not) helps the other person understand what you are trying to say?

The other day I realized that Net-Map is often just that, but taken to a higher level of general understanding and inviting others to co-doodle with you. By providing some basic steps to the doodling: first actors, then links, then motivations, then influence, Net-Map helps keeping the complex story on track and allows everyone to chip in and add their contribution.

As a facilitator some of my favorite Net-Map experiences (both with groups and individuals) were when the people I worked with just told their story like they would to a friend and I visualized this flow by writing the names they mentioned in the unfolding narrative on actor cards, sketching out the relations as they told me what happened. I think this is one of the reasons I enjoy Net-Mapping so much, because it can feel like you are just two people having a conversation – and not like being an interviewer who interviews someone or a person with a method which dominates the interaction (e.g. a closed ended questionnaire, where, every time the interview partner wants to tell you their view or experience, you have to say: “please just rate it on a scale from 1-5″. Or “possible answers are yes, no, don’t know”).

I guess that has something to do with respect: If I ask you to take some time out of your busy day to answer my questions, I want to show you I am really interested in your (own) answers and want to learn something I didn’t know before. I know that for a lot of quantitative analysis you need standardized questions and answers and it is great to be able to say something statistically significant about things… but I personally just prefer a situation where I can really connect with the other person and listen to what they have to say.

Online Ticketing powered by Eventbrite NodeXL Event at IFTF, Thursday, December 15, 2011
Along with the Social Media Research Foundation, the Institute for the Future is co-hosting a meetup for those interested in mapping social media networks. Users of tools like NodeXL and Gephi (among others) are welcome to join us for an evening devoted to collecting, analyzing, and visualizing social media networks. Thursday, December 15th at 6pm at the Institute for the Future‘s offices in Palo Alto at 124 University Avenue, 2nd floor.

Your email:

 

Online Ticketing for NodeXL/Social Media Network Mapping powered by Eventbrite