As I mentioned last week, I will be hosting videos of several of the keynote speakers from this year’s useR! 2010 conference at the video Rchive. As it happens, the first video I was able to upload was the panel discussion we held on starting local R user groups. I have uploaded the video, which is also embedded below (after the jump).

I was joined on the panel by an illustrious assembly of R community members, which included:

• Jim Porzak—San Francisco Bay Area
• David Smith—REvolution Analytics
• John C. Nash—Ottawa, Canada
• Szilard Pafka—Los Angeles
• Marck Vaisman—DC Metro
• JD Long—Chicago

Sometimes misunderstandings in conversations seem to be my main learning opportunity. Maybe that’s one benefit of working in a context where I am not a native speaker…  Today I presented Net-Map to my colleagues at AffinityLab and in the discussion afterward I said something about how great this tool would work for managing processes. What I didn’t realize was that a lot of people think of the following, when they hear process management:

A higher level, non-involved entity (for example “the” management) looks at problems from a social engineering perspective and comes up with a set of strict and not very useful rules that everyone else has to follow with the goal of standardizing processes – but often with the effect of increasing bureaucracy and decreasing motivation and problem solving capacity.

Net-Map is not a good tool to do this. And even if it was, I’d say: Don’t! Because, why would anyone want to do that???

So how can you use Net-Map as a process tool? What I was thinking of is this: At the beginning of a longer strategic process (maybe a project implementation, a product development or a organizational change process) you invite those involved and impacted to an initial Net-Map session to develop a baseline map and discuss the following questions:

• Where do we want to go from here?
• What do we need to do to get there?

You would discuss both your content goals and what strategic changes in the existing network might help you get there. Are there links that need to be strengthened or abandoned? Do we need to add more partners? What are coalitions, bottlenecks, potential and actual conflicts and what do we do about them? Who can do what to get us to a better future situation?

After that you go back to work and do whatever the purpose of your group or organization is. After some time, you get together again and draw a map of how it looks now: Some of your networking plans of the first round worked out and you see changes just like you predicted. In other areas achievements were more difficult. In the process you might have realized that some of your initial strategies were naive or counter-productive, that you didn’t understand the importance of some actors who became more central to the cause etc. Everyone involved was encouraged to adapt their strategies according to the learning that took place and in can explain in the second round of mapping how we got where we are and what we now need to do to get to the next level. While you compare the network plans with the actual network you have developed, this is not a simple assessment process that would focus on the achievement of pre-defined network goals.

If you start a process like that, you are saying: I trust my partners/employees that they are motivated to do their best and that together we can come up with better solutions than any individual could. But you also say: Let’s check in periodically to see if we are still on track and explore how we can think together and make sure we don’t get lost in networking for the sole purpose of networking.

Today, I am packing up the car and heading south to my old home, Washington, DC, for the useR! 2010 conference, which is being held at the National Institute of Standards and Technology. Incidentally, where I was an intern in the Information Technology Lab during college.

If you are not able to make the trip to Gaithersburg, MD; fear not, through the hard work of Szilard Pafka (organizer of the LA R user’s group) and Katherine Mullen, coordinator of useR!, I will be hosting many of the conference’s keynotes, lectures and several of the panel discussions at the Video Rchive. It may be several days before all of the videos are uploaded, but be sure to check back at the Rchive next week for any updates.

If you are attending useR!, do try to make it to our panel discussion on starting a local R users group in your area (Thursday, 3:25pm in the Red Room). The panel includes several prominent charactersmembers of the R community, and should be a very entertaining and informative discussion.

Hope to see you there!

After years of writings on the potential of using the Internet to improve democratic governance, one must think that citizens have numerous offerings to choose from. Lately governments have introduced numerous policy papers and declarations that put a priority on... Alexander Schellong http://www.citizen-relationship-management.de

As I have mentioned, I recently returned for a lovely trip to Europe. While on vacation my brilliant, beautiful, funny, and all around perfect girlfriend accepted my invitation to be my wife.

Pause for shared overwhelming feeling of joy…

While I am still basking in the glow of being the luckiest man on Earth, as a true data geek I could not let this opportunity to analyze a novel data set escape me.

One of the most fascinating aspects of social media is how it has changed the way life-milestones, like getting engaged, are announced. Facebook’s ‘Relationship Status’ feature allows users to inform all of their friends at once about these large life changes. Such announcements are often met with a sudden deluge of comments and wall postings, so I thought: wouldn’t it be interesting to collect this data and analyze the frequency of decay of these postings?

Though I am not on Facebook, my fiancée is, and with a little help from Facebook’s API and R’s ggplot2 library I was able to collect and analyze this data. Below I present (with permission) the data on from my fiancée’s wall for the first 48 hours after she changed her Relationship Status from ‘In a Relationship’ to ‘Engaged’.

Interesting. A huge spike in the first hour, a drop and flattening over the next two hours, and finally another large drop with sporadic spikes. Women dominate the initial posts, while the gender difference vanish as posting frequency decreased more late-comers make posts.

Of course, all of this is secondary to that fact that—Kristen—I love you and I cannot wait to spend the rest of my life with you!

If the level of online discourse is a good indicator of whether a topic has penetrated the collective nerd consciousness, then the notion of a burgeoning “data science” discipline has taken hold. A few weeks ago I discussed where to draw the line on this idea, but recently I again begann thinking about the idea and term more critically. Yesterday, I had a wonderful discussion with a brilliant member of the data community here in New York, which focused on the delicate balance between keeping a human-friendly face on mass quantities of data—something the data scientists are meant to do—and having this new discipline make formidable contributions to our general understand of human behavior.

That is, up to this point, many of the great evangelists of data science have focused on telling stories with data. Science, however, is not about story telling, but about discovery. Perhaps I am particularly cautious of the suffix “science” because of the awkward self-consciousness the word has imbued in my own discipline. At its roots, political science was a discipline that sought to construct narratives; equal parts history, philosophy and personal experience. The name “political science,” therefore, brought the ire of the “hard science” community, as they felt (perhaps with reason) that the word had been appended to the title erroneously, as there were no identifiably scientific aspects to the endeavor. While my discipline has come a long way in its application of the scientific method, and today can much more accurately be referred to as a science, there continues to be a delicate balance between discovery and story telling. What, then, can the data science community learn from this experience?

Broadly, all disciplines are measured by their contributions to our understanding of the universe. Data science—by design—is the product of measured human activity, and therefore should seek to provide new insight into human behavior. Unfortunately, the current focus of many of the community’s members has been a self-congratulatory appraisal of the tools that have been developed to allow for this large-scale measurement and recording. To be a successful discipline, however, the focus must move away from tools and toward questions.

To paraphrase a famous nerd, with great data comes great responsibility; so to begin, the data science community must ask: what questions do mass quantities of measured human existence allow us to address that were never previously possible? Just the thought should be enough to inspire some to begin writing research proposal, but in effort to contribute to this discussion here are a few things I hope data science will teach us:

• How do online discourses manifest in offline behavior? – I study terrorism, and one lingering problem in this area is the threat from so-called online radicalization. That is, to what extent does information obtained online influence individuals to join radical organization or commit acts of terror? This question, however, applies to many other areas, such as voting and purchasing decisions. As our ability to analyze these discourses increases, perhaps data science will provide some answers.
• How do we reach the “tipping point”? – Malcom Gladwell did well to introduce the idea of the tipping point, but since then we have learned too little about how these culminations occur, and what—if any—are the consistent behavioral features that lead to them. Often, these events occur online, where data science may be able to analyze the tracks that lead to these phase shifts.
• What are the ethical limits of personal data analysis? – The rise of massive stores of personal data online has been a boon to the data science community, but it has not come without some trepidation. With intimate knowledge of the tools and processes used to capture and analyze this information, this community is uniquely positioned to contribute to a discussion of the ethical limits of their own work.
• Do we really consumer things differently? – Everyday people make decisions about what they will consume; in terms of purchases, food, information, etc., and conventional wisdom states that these decisions are largely a function of birth cohorts, geography, educations, etc. Is this really case? The vast amount of consummatory data being generated online may be able to help us understand the most significant indicators of these differences.
• Can more/better data explain rational irrationality? – Today we learned some of the limits of behavioral economics, which have helped explain instances of seemingly irrational behavior. As the op-ed points out, however, there continue to be many questions that discipline fails to explain. Perhaps, then, the explanations of these anomalies can be borne out of data.

I welcome your own thoughts on what data science will teach us, and hope you will share them. Personally, I think this discipline has the potential to generate vast amounts of knowledge, but must be cautious to not loose sight of the question in the sea of information.

This paper focuses on how the method works, both for data collection and for facilitation of processes. If you are a regular at this blog, you know it all, but you’ll be glad to have it all in a condensed, quotable fashion.

“Net-Map: Collecting Social Network Data and Facilitating Network Learning through Participatory Influence Network Mapping”, by Eva Schiffer and Jennifer Hauck, Field Methods, August 2010.

Twitter is a gigantic repository for our collective state of mind. Every second, thousands of tweets reveal what everybody and their mother had for lunch, what Justin Bieber is up to, or what magnificent link you should be checking out... Sune Lehmann http://sunelehmann.com/

I have been back in the United States for about a week, but only now have found some time to get back to blogging. As I stated before my departure, the primary reason for my trip to Europe was to participate in the 30th meeting of the International Network of Social Network Analysts.

First, Aric Hagberg and I gave a workshop on using NetworkX to hack social networks. Given that it was the first time we had ever given this workshop, I was pleased with how well it went and the positive reception we received from the audience. It was encouraging to see so many researchers from academia, private corporations and the government interested in learning the mechanics of generating network data and analyzing it. That said, Sunbelt did reinforce my previous observation that academic researchers have a lot of catching up to do in terms of tools. There were several talks that indicated an unfortunate lack of technical expertise, which could easily be overcome with a minimal level of effort. Thankfully, conferences like Sunbelt allow for a people with many different talents to mix together and exchange ideas—and this norm was on display in Riva del Garda.Next, the panel session were a good mix of methodological research and substantive application. Despite some serious logistical impediments (heat, overcrowding, etc.), I was able to see some very interesting talks, and received some interesting feedback on my own research. Given the idyllic location of the conference, there were only a few times where most of the conference attendees were in the same place, which detracted from the networking opportunities—a somewhat shameful consequence, given it was a conference on social networking. The best interactions I had occurred while strolling the poster presentations. The biggest winners of the poster sessions were the dynamic duo of Mathieu Bastian and Sebastien Heymann, the purveyors of Gephi. We discussed potential future opportunities to interface Gephi with NetworkX, but Gephi itself was a huge hit at Sunbelt and I expect to see its beautiful graphs on display in future network papers.

It appears that Sunbelt XXXI will be in St. Pete Beach, Florida, and I hope to see an even larger crowd there in 2011. By way of closing the book on one networks conference and opening it on another, I would also like to pass on an announcement I was sent about an upcoming conference on network visualization. In October, Harvard will be hosting the Connecting the Dots symposium, which will feature keynote talks from Alessandro Vespignani and Ben Fry. The conference itself looks interesting, but there is also a very inclusive call for presenters:

In addition to the keynotes, we are soliciting proposals for guest speakers to give short 20-minute presentations. We are interested in any presentation that includes the visual depiction and/or visual analysis of network data as a central theme. Potential topics include but are not limited to network visualization algorithms, network visualization software, network communities and visualization, other network theory or analysis, and artistic projects centering on network visualization. Given the cross-disciplinary nature of network science, we welcome applications from researchers in any scientific discipline.

Seems like a great opportunity for anyone studying network and network representation. I expect to see my friends from Gephi there, and hope others will submit.

Photo: napolipuntoacapo.it

I know, for many of you that’s nothing to write home (or a blog post) about and for some it might be just the daily drag… But after two years of working from the sofa-bed-kitchen-table-home-office, let me tell you, having to get properly dressed to go to work feels great. And having colleagues. But no boss. What better workplace could there be?

As of this month I am a member of the Affinity Lab, a shared workspace, where people like me (and maybe like you) can rent desk space at affordable rates and work in a room full of people like me. Or, even better, unlike me. Because we know from social network analysis 101 that heterogeneous networks are best for innovation. Ok, you need a bit of similarity as well, otherwise it’s difficult to actually develop and maintain connections. So when I first came here and told the manager Phillipe Chetrit, what I do for a living, he was excited. He immediately got why it is interesting, without knowing yet how it works. Next week I will give a brown bag seminar for my co-workers here and see who shares the excitement even after understanding how it works. So, if you work from home and you are ready to strangle your cat, geraniums or husband, I can highly recommend finding a little desk space somewhere away from cats, geraniums and husband.

I wonder if a Goldman Sachs executive would spend a night in the house above? After all, they do kind of "own" it. The house, in the Slavic Village neighborhood of Cleveland, had a mortgage that was part of the Goldman Sachs synthetic CDO, ABACUS 2007-AC1. In 2008, the house was foreclosed upon, and still sits abandoned today.

At least the ivy devouring the south side of the building seems happy.

I am writing this in the Red Chimney restaurant, a social hub for the locals in Slavic Village. This place is probably the only thing that has not changed drastically in the last ten years in this neighborhood. It feels very comfortable.

For more details, diagrams, and how we connect the dots between Goldman Sachs and this gold-less neighborhood, see our orgnet.com web site.

Update: Mortgage fraud via "flipping" happens in good neighborhoods also

I wonder if a Goldman Sachs executive would spend a night in the house above? After all, they do kind of "own" it. The house, in the Slavic Village neighborhood of Cleveland, had a mortgage that was part of the Goldman Sachs synthetic CDO, ABACUS 2007-AC1. In 2008, the house was foreclosed upon, and still sits abandoned today... the ivy devouring the south side seems happy.

I am writing this in the Red Chimney restaurant, a social hub of this struggling neighborhood. Motioning to my iPad, the waitress comments, "We don't get many people in here with those." The restaurant is half-full with locals, most over sixty. This place is probably the only thing that has not changed drastically in the last ten years in this neighborhood. It feels very comfortable. Four bikers walk in, no one bats an eyelash.

For more details and interesting maps, we connect the dots and show the flows, between Goldman Sachs and this gold-less neighborhood, over on the orgnet.com site.

The Journal of Social Structure has released its First Annual JoSS Visualization Symposium results and two of the images were generated with NodeXL.  One of the two is Bernie Hogan’s radial layout applied to representing Facebook Friend networks.

http://jossviz.wordpress.com/2010/06/23/friendwheel-layout-of-a-facebook-network/

The Journal of Social Structure (JoSS) is an electronic journal of the International Network for Social Network Analysis (INSNA).  Here is Bernie’s description of the graph.

This is a “pinwheel” diagram using the author’s Facebook personal network (captured July 15, 2009).

Nodes represent the author’s friends and links represent friendships among them. The author is not shown. Each ‘wing’ radiating outwards is a partition using a greedy community detection algorithm (Wakita and Tsurumi, 2007). Wings are manually labelled. Node ordering within each wing is based on degree. Node color and size is also based on degree. Nodes position is based on a polar coordinate system: each node is on an equal angle of n/360º with a radius being a log-scaled measure of betweenness. Higher values are closer to the center indicating a sort of cross-partition ‘gravity’.

This layout has several notable features:

- The angle of each wing is proportionate to its share of the network. Thus 25 percent of nodes go from 0 to 90º.

- Partitions are distinguished by their position rather than a node’s color or shape.

- The tail indicates the periphery of each partition. A wing with many tail nodes indicates many people who are only tied to other group members.

- Edges crossing the center show between-partition connections. Since nodes are sorted by degree it is easy to see if edges originate from the most highly connected nodes or the entire partition.

Bernie’s chapter on analyzing Facebook networks with NodeXL appears in the book: Analyzing Social Media Networks with NodeXL: Insights from a connected world.

Share and Enjoy:

There is a Stanford Media X event on July 22nd, 2010 on new tools for SNA:

Network Analysis Made Easy:  Using NodeXL To Map Social Media Networks

http://mediax.stanford.edu/WSI/marc.html

Bring a laptop (running Windows and Office 2007 or 2010) to this workshop and you can be analyzing a social media network from systems like Twitter, flickr, YouTube and your own email by the end of the day.  If you can make a pie-chart in Excel, using the free and open NodeXL (http://nodexl.codeplex.com) you can now make a rich network graph from data extracted from social media systems and other common formats.  If you have a network, bring it, if not you can bring a suggested topic that we can map during the course of the day.

Even if you leave your laptop behind or have a Mac (sorry, no version is yet available for MacOS – unless you have a virtual machine with Windows and Office) this workshop will introduce the core concepts of network science with application to social networks in general and social media networks in particular. Applied to a range of topics and services, social media network maps can illuminate a variety of “publics” – populations who share a common interest and may share connections.  Maps of topics like “oil spill”, “global warming” and other issue and event related keywords can reveal the groups and factions that cluster around different concepts and terms.  Key contributors in these maps can be identified through the application of network measurements that capture various aspects of a  person’s location in a network graph.

Share and Enjoy:

Faculty Summit

The 2010 Microsoft Research Faculty Summit was held July 12 and 13 in Redmond, Washington.  Among the many panels and discussions related to the state of computer science the NodeXL team had several representatives talking about the ways network science education can be expanded using an easy to use application for network analysis built on Excel.

Jimmy Lin from the University of Maryland also attended to speak about programming in the cloud.

Here is the abstract for the NodeXL talk:

NodeXL – Social Network Analysis in Excel—Natasa Milic Frayling, Microsoft Research; Ben Shneiderman, University of Maryland; Marc Smith, Connected Action

Businesses, entrepreneurs, individuals, and government agencies alike are looking to social network analysis (SNA) tools for insight into trends, connections, and fluctuations in social media. Microsoft’s NodeXL is a free, open-source SNA plug-in for use with Excel. It provides instant graphical representation of relationships of complex networked data. But it goes further than other SNA tools—NodeXL was developed by a multidisciplinary team of experts that bring together information studies, computer science, sociology, human-computer interaction, and over 20 years of visual analytic theory and information visualization into a simple tool anyone can use. This makes NodeXL of interest not only to end-users but also to researchers and students studying visual and network analytics and their application in the real world. NodeXL has the unique feature that it imports networks from Outlook email, Twitter, flickr, YouTube, WWW, and other sources, plus it offers a rich set of metrics, layouts, and clustering algorithms. This talk will describe NodeXL and our efforts to start the Social Media Research Foundation.

Some photos from the event:

Saul Greenberg

Ben Shneiderman and Andy van Dam

Ben Shneiderman, Natasa Milic-Frayling and Marc Smith

Tom McMail and Marc Smith

Share and Enjoy:

The NodeXL team has just released a new version (v.1.0.1.128) that contains a new “Automation” feature that allows users to define a collection of operations to perform on their network graphs and invoke the complete set in a single button click AND reuse that configuration on other workbook graphs.  In fact, the feature will apply the configuration you define to all the files you specify, allowing easy processing of large collections of network data sets.

This week the feature is partially complete.  Users can invoke the merge duplicate edges, calculate graph metrics, auto-fill columns, create sub-graph images, find clusters and show graph.  These operations can require as many as dozens of clicks when performed manually.  If you have dozens or hundreds of network data sets the result is a daunting case of repetitive strain injury and carpal tunnel syndrome.  Instead, with automation, these operations can be carried out orders of magnitude more frequently without much pain!

The next release will feature the complete package which will then include control over the layout and graph options.  As a result, automatically generated network visualizations can be produced in a pipeline: users will be able to specify a query using the NodeXL desktop network data collector and then automate the processing of  large collections of data sets.

The result should be better analysis of time series data sets that have many “slices”.  The feature points the way to additional development work for supporting the comparison between networks to evaluate their evolution.

The REM album “Automatic for the people” takes its title from the motto of Athens, Georgia, eatery Weaver D’s Delicious Fine Foods.

Share and Enjoy:

A new paper on visualizing social media has been released on the University of Maryland, Human Computer Interaction Laboratory tech report archive.  Co-authored by Derek Hansen,  myself, and Ben Shneiderman, the paper describes and visualizes the patterns of connections formed when people tweet about events like conferences and news stories.

EventGraphs_2010_HCIL_Tech_Report

http://www.cs.umd.edu/localphp/hcil/tech-reports-search.php?number=2010-13

Hansen, D., Smith, M., Shneiderman, B.
EventGraphs: Charting Collections of Conference Connections
HCIL-2010-13

EventGraphs are social media network diagrams constructed from content selected by its association with time-bounded events, such as conferences. Many conferences now communicate a common “hashtag” or keyword to identify messages related to the event. EventGraphs help make sense of the collections of connections that form when people follow, reply or mention one another and a keyword. This paper defines EventGraphs, characterizes different types, and shows how the social media network analysis add-in NodeXL supports their creation and analysis. The paper also identifies the structural and conversational patterns to look for and highlight in EventGraphs and provides design ideas for their improvement.

Share and Enjoy:

Four years ago, Frederick Bieber, Charles Brenner and I wrote a paper in Science on the feasibility of "familial searching" of offender DNA databases for leads. Familial searching utilizes the known statistical correlations in the genetic profiles of close relatives... David Lazer

I’m working with Alive & Thrive, a Gates Foundation funded project to improve young infant and child nutrition in a number of third world countries. Net-Map is part of the monitoring and evaluation component that is led by IFPRI. The question we are looking at at the moment is: If the project aims at changing the networks related to infant nutrition, how can we monitor if it actually does. And who do we need to ask to figure this out.

The initial intuitive approach is to ask those people who are knowledgeable about the issue and the existing networks now and ask them again towards the end of the project. But looking at it more closely I realized: One of the goals of the project is get groups and individuals involved who are not interested in the issue as of now. Sure, if you ask the experts now and in three years, they might say that these marginal actors have become more involved. But you would get a so much brighter picture if you ask the marginal actors now (when they have a rather fuzzy vision of the network and place themselves at the fringes) and after the intervention (when, hopefully, they know much more about how the network works and put themselves in a more defined position). So even though the before interviews with marginal actors will be confusing, not very efficient and lead to little reliable data about how the network works at the moment, you need exactly these unclear pictures from the beginning of the project to be able to show afterward that you did have an impact on their involvement and network knowledge. And if these interviews are additional to the core actor interviews, you will still get a pretty good picture of the before and after network as a whole.

In my typical “learn more about a network in short time” projects I would recommend only interviewing people who are knowledgeable about the network and stop the snowballing when the answers start getting boring (saturation point). But for monitoring and evaluation purposes I might have to re-think this recommendation. So my working recommendation now would be: Interview some overview experts / highly involved actors and interview those whose network position and network perception you want to change. Get a combined network picture for before and after from your highly involved actors. But also compare the individual before and after networks of your target population.

The International Sunbelt Social Network Conference is the official conference of the International Network for Social Network Analysis (INSNA).

This year’s INSNA “Sunbelt” conference is at the  Riva del Garda Fierecongressi, Trento, Italy!  Here is the 2010 INSNA Sunbelt Program.

This is the NodeXL map of connections among people who tweeted the hashtag used for the conference “#sunbelt”.

Having now seen several of these maps for other topics and events (see: http://www.flickr.com/photos/marc_smith/sets/72157622437066929/) this map can be placed in context.  It is a small group, but has a high density of connections.  It lacks isolates, the people who say the term but do not connect to others who say that term.  This means that this is a very “in-group” population: if you know to use the #sunbelt hashtag, you probably connect to someone else who uses the term.  It is a single major cluster of connected people, no obvious sub-graphs or clusters are visible.  Not everyone is central in the graph, and those who are have a prominent role in the network science community.  Here is the top ten list of #sunbelt mentioning twitter users ranked by betweeness centrality.

miriamnotten
barrywellman
memeticbrand
isidromj
drewconway
gephi
kristtina
danevans87
valdiskrebs
ciro

Share and Enjoy: