Alchemical Musings

Originally published on theploneblog.org

Preliminary report on the Big Apple Sprint

July fourth has come and gone, but the fireworks set off at last week’s sprint are still visible.

The sprinters arrived at Columbia University bright and early, Wednesday morning. (note to all future sprint organizers: tell the caterers to skip the decaf and double the regular order). About ~13-15 sprinters were present, but we also coordinated remote sprints with Austria (+5 hours ahead) and Utah (-2 hours behind) meaning we were basically sprinting around the clock.

We all used the freely available, plone-based, OpenPlans service to manage our collaboration and everyone found the software to be extremely reliable and easy to use.

The sprint began with introductions and detailed demos of the tools and
products people had been working on and were most proud of. Sprints are
difficult to plan in advance since the skills and interests of the
attendees are not decided until the final roster shows up. A diverse
range of interests were represented, but common themes rapidly emerged
– most of us were working at or with non-profits, building community sites with educational goals,
and we all had a very strong interest in participatory media (in contradistinction to mass broadcast media). It became
apparent that although we were all working on different problems, very similiar features and tools could be part of our solutions.

The sprinters self-organized into 3 teams (+ 1 remote team):

Multimedia – focusing on improving the handling of multimedia
content w/in Plone. Topics included transparent management of multiple
media formats, improving the quicktime player, abstracting the common controls from the different media player formats, merging Austria’s
ATVideo bittorrent branch, allowing for remote resources to be managed by the
media types, and the integration of CCNMTL’s video clipping tool into
PloneMultimedia. Thanks to Nate, Gary, Anna, Kurt, and Sky for making this group a productive success.

Discussions emerged around the hybridization of modern media formats.
Is an audio track with synchronized gifs a piece of audio or video media? Modern events are now distributed in multiple media
formats (e.g. podcasts and vodcasts) – might PloneMultimedia benefit from a
UI analogous to PloneLingua? What about metadata embedded directly in
the binary file? Should it be editable ttw? Which file is the
canonical file and which are the derivatives? Whatsoever will we all do about large file support?!?

A new ‘media’ container was introduced to PloneMultimedia allow for the mgmt of media that spans multiple traditional formats.

Annotations/Tagging – laying out the jigsaw puzzle that tagging,
rdf, taxonomies, folksonomies, and sticky notes, and microapps have become in the hopes of
consolidating on a common strategy to move forward. The Yucca project was born
after we all began to realize how many of our problems would naturally fall into place with a robust engine which supports user contributed content annotations.

Also, work was done by Anders and Chad on the sticky notes product with the aim of
factoring out the notes so they could be used outside of plone too
(with the persistence abstracted, so that it could be backed by a
microapp – like pita, or even stored client side), as well as improving the
editability of the notes – they now support “double-click to edit”.
Great job!

Blogging/Syndication – This group (Rob and Kurt) was primarily working on polishing quills so that it provides a smooth user experience.
Progress continues and Quills is looking like a serious contender.

Content Licensing – see Nate’s post on conetent licensing in plone. This work was conducted primiarly by the group working in Utah, out of C()SL.

In case its not obvious, there was a great deal of overlap between the interests of the groups. For example, the multimedia team was also very interested in tagging, syndication ((p/v)odcasts), and licensing. Although it initially seemed challenging to tease the participants apart, the groups self organized quite organically.

Some time early in the sprint, Rob treated us all to an impromptu tutorial on z3 annotations and continued to support the various groups in their attemps to absorb the new world order. By the end of the day, 3 seperate pairs of developers had the contentratings product installed and running, demonstrating how important it is for there to be good examples representing best practices in the collective – these practices spread virally. Whit actually spent a chunk of time with Christof and Darian thinking about a good packaging system for keeping the z2 and z3 layers as thin as possible. They also spent time backporting the tagger product from zope 2.10 to  2.9.

Throughout the sprint curious academics wandered through in an attempt to catch the rare glimpse of a geek working in the wild.

Ian Bicking also stopped by to check in with his new coworkers.

Since 3 days of sprinting is never enough, sprinting actually continued over the weekend at the openplans offices with a reduced number of hardcore plonistas. Christoff was gracious enough to invite us into his home for a home cooked meal.

If anyone has anything else to add to this report, please do – sorry I missed it.

The sprint was very productive, educational, and great fun as well.
Beyond the technical achievements, relationships were forged that we
expect to flourish in the months to come. I think we all witnessed
tremendous convergence across our organizational requirements, and are
also convinced that the tools we are working on will be in great demand
once the corporate world figures out how useful these technologies can
be.

Throughout the sprint participants shared tips and tricks,
demonstrating how important live, in-person communication remains, even
in the Google Age. This was especially apparent when the nyc sprinters
attempted to convey proceedings to the remote sprinters and we felt the
limitations of geographic displacement.

Sky got married on Sunday in a park on the Hudson river. His last days as a
bachelor were devoted to this Sprint, and we all wish him and his new
wife a happy and healthy (Plone-free) honeymoon.

Pictures

“bigapplesprint” on flickr.com

And, since Anders is so special, here is his set:

Special thanks to our hosts, Columbia’s Center for New Media Teaching and Learning for sponsoring the sprint and making us all feel at home in New York City.

Filed by jonah at 10:01 pm under plone
No Comments

Originally published on theploneblog.org

A forray into drupal 4.7’s taxonomy system and what Plone can learn from it.

I have been moonlighting on a Drupal project and paying close attention to their taxonomy system. Drupal’s taxonomy/category/tagging system was completely revamped for their 4.6->4.7 release – a release close to a full year in the making, analogous to the Plone 2.0->2.1 “minor” point release.

The site I have been working on, theicarusproject.net

has a very rich collection of content, and one of the primary motivations for the migration is to get a better handle on the classification system – noboday can find anything on the current site. They were committed to Drupal long before I arrived, so I dug in with the hope of learning something from the contrast.

PHP bashing aside, there are alot of interesting things happening in Drupal land. I hope to follow up this post with a few more cross-pollinating nuggets, but for now I will focus on their taxonomy system.

Taxonomies in Drupal are considered the heart of the system, and the essential modules ship with the core and cannot be disabled. Most URLs in Drupal are effectively queries, much like our smart folders (actually, for anything aside from anti-chronological display order you need to install the Views module) but the display results are all instances of content with matching vocabulary terms. The absence of folders and containment initially confuses many administrators, and renders breadcrumbs largely useless, but does allow for the creation of sophisticated information architectures.

Taxonomies are managed top-down, not bottom-up, and have a separate administrative interface for their creation and management. Once the taxonomy vocabularies are created, specific terms can be added to these vocabularies without having to create content associated with those terms (in contrast to a bottom-up category system, like the mediawiki).

Category Management – Vocabulary Listings:

Druapl supports multiple vocabularies, which can each be associated with one or more content types. Vocabularies can be flat, one level deep, or N-levels deep (hierarchical). They can be fixed or free form (meaning content authors can make up new categories upon content creation). The core tagging system does not support the creation of tags per-user, per-object – only per-object.

Category Management – Add a Vocab:

Category Management – Add/Edit a Term:

The Drupal taxonomy system is very powerful, but its power is very open ended and does not necessarily lead users towards a uniform experience. The confusion around categories and taxonomies is best exemplified by the category module meant to consolidate and simplify taxonomy and navigation, but there is no consensus on its incorporation into the core.

A large number of modules are built around taxonomies. Core Drupal supports roles, but no groups (organic groups is a popular access delegatoin solution, but it is incompatible with other access restriction modules – so you have to choose one), and does not have a notion of containment (ie folders). So, for example, one way to restrict editing access is by enabling the taxonomy access module. Another useful module is the taxonomy browser which allows for advanced search against unions/intersections of vocab terms.

Category Browser:

Once vocabularies are created, and terms added, content can be associated with these terms:

Working on this site really drove home the value in separating the navigation axis (section) from the thematic axis (keywords), and separating these dimensions was easy to accomplish with the taxonomy/category tools built into drupal. In particular, once the scheme was developed, managing vocabulary lists (even hierarchical ones) is intuitive, albeit slightly clunky. I further chose to introduce a free-form tagging dimension for member contributed posts which may or may not fit into the fixed taxonomy. This is similar to myspace and facebook allowing for free-form hobbies and interests, and banking on a large enough user base that there will be overlap and potentially interesting intersections.

The system still does not allow for the intuitive modeling of a many-to-many relationship, which I continue to think is the litmus test which will mark of a truly powerful taxonomy UI. There is still quite a bit of programmer know how involved in setting up this system so that it operates the way that content administrators expect, and arguably there are too many degrees of freedom introduced by such a general purpose modeling capability (if you think about it, a tagging system can essentially allow web administrators to model relationships which used to require programming custom applications against an rdbms).

Nonetheless, Drupal’s taxonomy/category/vocabulary system definitely captures a few use cases more elegantly than Plone’s current core does. But perhaps the real lesson is the importance of not mixing navigation space and content space, which can be kept separate in Plone, but is all too easy to conflate (in Drupal too!).

Note: most things I describe in this case study could have been accomplished within core Plone – I think the most interesting things here are the administrative UI for multiple vocabulary management, the different types of vocabularies, and how central they are in the construction of a Drupal site.

Filed by jonah at 9:41 pm under plone
No Comments

Seymour Papert , the inventor of Logo, spoke at Teachers College on Monday April 10th. I was lucky enough to hear him talk in a standing-room-only event. My former employer, Idit Caperton
studied with Papert, and MaMaMedia incorporated many of the principles he advocated.

His ideas, once stated, are remarkably simple and obvious–usually a mark of the good ones. He thinks we are teaching mathematics ass-backwards, and that we ought to introduce it the way it came about in the history of humanity – engineering first. This approach will create and foster the demand for mathematics. Pyramids, navigation, astronomy, all drove the development of mathematics – and robotics and programming can provoke and instigate the need for mathematical abstraction in education. Sounds about right.

Interestingly, his experiments have led to anecdotal accounts of a reversal of the gender discrepancy in science/math. He claims with an engineering first approach, girls actually quickly excel beyond the boys, venturing beyond speed and destruction to the mastery of a much wider variety of skills with the systems.

He also demonstrated, in 10 minutes flat, how logo can be used to teach 2nd graders the notion of a mathematical theorem (in creating any closed shape, the turtle will rotate through a full 360 degrees – repeat N {fd 10 rt 360/N}) as well as how to introduce calculus (through the idea of the limit). He made the point that once a second grader is arguing — “that’s not a circle, its lots and lots of short lines”, you have already won…

If logo has a failing, its that it does not provide the necessary scaffolding for teachers other than Papert to effectively teach with it. I have been exposed to logo in the past, but never really understood its appeal until Seymour started turtling.

Interestingly, Logo is far from irrelevant. Mark Shuttleworth’s ClassroomCoders curriculum imagines a logo->squeak->python pipeline for educating the programmers of the future…

Seymour is also heavily involved in the $100 laptop project, a project which many consider to be one of the most important educational initiatives currently underway.

Filed by jonah at 11:35 pm under epistemology,water
1 Comment

April 7th I heard Lev Manovich talk at Pratt. I am a big fan of Manovich’s written work, and the Language of New Media was instrumental in my analysis of tagging.

Friday night Manovich showed us ideas in progress, and bravely admitted that they were not completely formed. He talked about describing the evolution of media in evolutionary terms. As in, the next logical progression after getting all our media digitized (i.e., simulating physical processes w/in the digital environment) is the breeding and hybridization of the media. He is claiming that some of what we are now seeing in ‘moving graphics’ or ‘design cinema’ is actually a new form of media, distinct from what came before it. And he is interested in identifying the trunks and branches of this media evolution.

Plaid Itsu was a film he used as an example of a completely new form. Whereas multimedia was the assembly of multiple forms of media adjacent to each other, metamedia is the combination of these forms into a new unified whole. He pointed out the live action photography, combined with traditional design aesthetics, combined with graphics, etc etc. Not sure I bought it, but it was an interesting assertion.

The best question from the audience alluded to a longstanding disconnect between media and communication theorists. Manovich is looking exclusively at the end product of the media being created, and not examining the cultural and social conditions that lead to its creation. There may be mileage from this rarefied approach, as some patterns are discernible, but it does seem to be lacking the depth to explain the creative dynamics and underlying motivations.

After the talk, I began to this relate his line of reasoning to Arthur Young’s theory of process:

The Theory of Evolutionary Process as a Unifying Paradigm
Theory of Process Poster (too bad this isn’t really visible online)

Which I first became exposed to through the work of the Meru Foundation:
letter matrix

It seems to me that the evolutionary forces that Manovich is documenting conform to the trans-disciplinary evolutionary process that Young articulated. For what its worth, the hybridization of media that Manovich claims we failed to predict, was foretold back in this book on the MIT Media Lab, published in 1988.

Filed by jonah at 11:34 pm under aesthetics,epistemology,fire
No Comments

Last weekend’s Cultural Studies conference reminded me of a viscous cycle that many humanities-oriented researchers are being subjected to. Disciplines such as educational research, ethnography, anthropology, cultural studies, sociology etc have effectively been colonized by the methodology of the social sciences and they are being forced to play a numbers game which they may not be suited for.

Many projects striving for credibility are subjected to the tyranny of statistics – forced to transform their qualitative information (interviews, transcripts, first person accounts) into quantitative information through the process of coding. This reduction forces the data into buckets and creates a significant degree of signal loss, all in the name of a few percentages and pie-charts.

Perhaps we have lost sight of the motivation for this reduction – the substantiation of a recognizable, narrative account of a phenomena, supporting an argument. Arguably, the purpose of the number crunching is to provide supporting evidence for a demonstrable narrative. Modern visualization techniques may be able to provide one without all the hassle.

True, this is not always the only reason that qualitative is transformed into quantitative data, but advanced visualization techniques may provide a hybrid form that is more palatable to many of the researchers active in this area, and is still a credible methodology. It seems as if many people are being forced into coding and quantification, when they aren’t thrilled to be doing so. But the signal loss that coding is responsible for, all in the name of measuring, might be unnecessary if people think about using data visualization tools, that comprehensibly present the data, in all of its richness and complexity, as opposed to boiling it down to chi-squared confidence levels (and does this false precision actually make any difference? Does a result of 0.44 vs. 0.53 tell significantly different stories?)

In a thought provoking post on the future of science, Kelly enumerates many of the ways new computing paradigms and interactive forms of communications might transform science. The device that I am proposing here might lead to some of the outcomes Kelly proposes.

For a better idea of the kinds of visualization tools I am imagining, consider some of the visualization work on large email corpora coming out of the M.I.T. media lab, or the history flow tool for analyzing wiki collaborations, but even the humble tag cloud could be adapted for these purposes, as the power of words and visualizing the state of the union demonstrate.

Crucially, tools analogous to Plone’s haystack Product (built on top of the free libots auto-classification/summarizer library) might help do for social science research what auto-sequencing techniques have done for biology (when I was a kid, gene sequences needed to be painstakingly discovered “manually”).

The law firms that need to process thousands of documents in discovery and the commercial vendors developing the next generation of email clients are already hip to this problem – when will the sciences catch up?

For any of this to happen the current academic structure needs to be challenged. The power of journals is already under attack, but professors who already have tenure can take the lead here and pave the road for their students to follow.

Filed by jonah at 11:25 pm under air,epistemology
2 Comments

Today I presented last year’s bioport Part II paper to the 2nd annual Cultural Studies conference at Teachers College.

Permanent Records: Personal, Cultural, and Social Implications of Pervasive Omniscient Surveillance

I think the distilled version of this model if far more digestible and accessible than the papers.

One of my co-panelists is doing some really interesting work with urban
youth in the bronx, and gathering incredible interview materials about
the perceptions of surveillance by these youth, and their forms of
resistance. These stories might help convey the violence of a
surveillance society.

The conference format was a bit disappointing. I can barely believe academics still read their papers to each other at conferences – there are so many things that Open Source does right, including, knowing how to throw a great conference. Even the variety of presentation formats is an idea that needs to spread – BOFs, lighting talks, presentations and posters all create different spaces and dynamics for interactions between participants. The traditional model is so intimidating that it seems like many people are discouraged from participating.

More importantly, the social justice issues and governance models that are being explored by F/OSS projects are really important for the Cultural/Critical studies folks to be considering. It is also shocking how disconnected they are from the freeculture movement, and its theoretical roots. Arguably, the freeculture movement is a shadow struggle, mirroring the struggles for sustainability, and against globalization and the logic of capitalism being conducted in the physical world. But, it may also represent the actual ground on which that struggle is being conducted.

Filed by jonah at 1:13 am under air,fire,metaphysics
4 Comments

Originally published on theploneblog.org

Do not try to bend the folder — that’s impossible. Instead, only try to realize the truth. Then you will see that it is not the folder that bends–it is only yourself.

Tagging seems to have spurred a growing amount of research on categories and classification.

A recent paper by Clay Shirky, entitled Ontology is Overrated: Categories, Links, and Tags directly challenges the desktop metaphor which currently underlies much of Plone’s UI. To be sure, it is certainly possible to model the connections that Shirky describes using topics, smart folders, and a disciplined use of keywords, but the metaphor is critical for designing and intuitive system, all the way down to the icon.

I have recently been working a bit with Drupal, and their handling of this problem is worth checking out.  The core taxonomy module, combined with its corresponding menuing systems (menu, sitemenu, taxonomy menu) provide a great deal of flexibility in this regard.

Organization is going organic.

Filed by jonah at 9:34 pm under plone
No Comments

Two months back I saw Richard Stallman talk at a NYC Gnubies event and I asked him a question that I have been thinking alot about lately — Would a Saint in the Church of Emacs use gmail?

To me the question revolves around the growing threat that 3rd party webservices poses to the freedoms that free software is designed to protect. In O’Reilly’s What is Web 2.0 he argues that software is transitioning from an artifact to a service, and that data is becoming the new “intel inside”. In an age when applications have become commodities, could the freedom of my data (in an open format) be interchangeable with the freedom of software?

I recently listened to the Chief Open Source Officer at Sun Mircosystems pose a similar question in his talk, The Zen of Free. He talks about the importance of Open Software implementing Open Standards, which is close to the idea I have been advocating, but doesn’t quite go far enough.

Using free (as in beer) third party web services is very tempting, but I am worrying more and more about the traditional freedoms that free software protects against – vendor lock-in, proprietary data formats, and freedom to modify policy according to application specific requirements.

I would be less antsy about using web 2.0 apps if I had some assurance that I could get my data back out without screenscraping a bunch of html pages. Even services with APIs like flickr and delicious create vulnerabilities, as I was loathe to discover last week. Delicious provides a programmers api, but its api only exposes methods which operate on a single user. Thus, if you want to export a collection of links that have all been tagged with a particular tag, (reasonable if you are engaged with a community in distributed research) you are back to screenscraping!

These considerations and more advocate for the need for free (as in speech) versions of many of these services. There are certainly some side-effects of running a centralized service that are inherent in it being centralized, but many communities are making use of these “public” services because of their convenience, and the ease with which they can be “mashed up.”

Which brings me back to the design that we have been thinking alot about at work lately. Anders and I presented a talk at pycon demonstrating some of these ideas. Anders did a great job writing our talk up here:

Tasty Lightning

Crucially, it is imperative not to conflate our advocacy for building components that expose themselves as webservices with building apps against third-party web services. The design we describe resembles a traditional mash-up, except the components involved are locally controlled as opposed to relying upon external, corporate services. For all the usual f/oss reasons it can be important to “own” and run your own services.

But this argument also has everything in the world to do with Ulises In Defense of the Digital Divide as Paralogy essay. In this essay Ulises grapples with Lyotard’s critique of new media under the logic of capitalism which has “established commodification and efficiency as the ultimate measures of the value of knowledge.”

he continues:

…Lyotard states, in the final passage of The Postmodern Condition, that new media technologies can be more than simply tools of market capitalism, for they can be used to supply groups with the information needed to question and undermine dominant metaprescriptives (or what might be called ‘grand narratives’). The preferred choice of development, for him at least, is thus clear: ‘The line to follow for computerization to take . . . is, in principle, quite simple: give the public free access to the memory and data banks’ (Lyotard 1984: 67). (Gane, 2003, p.9)

Considering Google’s stated ambitions to “house all user files, including: emails, web history, pitcures, bookmakres, etc” the freedom movement better wake up to the fact that there is more to freedom than free software, and we are being outflanked.

Free software is only one corner peice of this puzzle – to complete the jigsaw we need the corners of free data, in a free format. Anything else?

(yes, I know I am posting this question using blogger – a situation I hope to remedy after the semester finishes).

Filed by jonah at 11:24 pm under Uncategorized
5 Comments

isomorphic surprises: stickies, tasty, and the importance of user contributed content

I have been thinking alot about tagging lately, especially how a complete tagging system – comprised of user-item-tag triplets, is isomorphic to rdf‘s subject-predicate-object triplets. It is amusing to think about how egocentric Web 2.0 is – The subject is always me. Web 2.0 might be made of people, but not just any people – Just the most important one in the world.

Today I had a fun time trying to explain to people on irc the power and importance of user contributed content annotations within plone. Crucially, user content annotations are per-user, per-object, and in many cases a single user might want to annotate a particular object with more than one annotation.

Interesting annotations can come in many flavors. There are free form notes, fine grained annotations (anchored to particular phrases – think msword trackback – or geometrical coordinates on the target – nice for image annotations), keyword annotations (aka – tags), etc ect. There many problems that can be solved with custom per user content annotations, including quiz and poll results, per student answer submission, and lately we have been working on allowing users to clip audio and video by annotating start and end times on media objects.

It is important not to confuse per-user tagging with DC:Subject – the dc metadata is shared across all users (like categories in the wikipedia) and in that sense, is global. While we are on the topic of tags, it is useful to talk about the vocabulary that drives the tagging. Vocabularies can be fixed or free, individual or collaborative, and personal or shared. All of these variations are interesting in different cntexts, and have to do with whether or not I see your tags, or if we each are developing our own ontologies.

In educational technology annotations are a big part of the problems we are trying to solve, but there are tons of use cases in the world at large. Additionally, a high performance, robust tagging engine can power personal content organization, like gmail’s labels.

Which brings me to the products we have been developing at CCNMTL. We have been using PloneStickies, a general purpose content annotation framework, in production for over a year now. Built using AT References, it allows us to create per-user AT objects connected to the target object. AT Schema Annotations won’t do the trick here since, like DC:Subject, these annotations are instance-wide. Z3 annotations might work, but by building AT derived stickies, we pick up search, workflow, permissions, and the richness of AT. This allows us to quickly and easily develop custom stickies, like the StickyClip.

PloneStickies has a ways to go, but the basics work great. It is not yet super useful out of the box, since the portlet it ships with only allows users to attach a single free form StickyNote to the target object. But it is great to develop applications with. It ships with with some super snazzy css stickies, complete with colored/resizable/title-barred/drop-shadowed/roll-upable/transparent-when-dragging notes, which can preserve their own x-y position and state across sessions and never fall off your screen. It now supports attaching multiple stickies to a target, but does not yet provide a mechanism for the target object to place the stickies itself.

At first we thought we could implement a Plone tagging solution using this framework – just create a StickyTags made out of keyword fields, and voilà – plonr. Trouble was, since tagging is such a symmetrical model, its tough to build an efficient zodb implementation (for me, at least) that can handle all the querrying we wanted to do.

Enter the tasty microapp and the PloneTasty proxy (about 90% done). Tasty is a stateful (sqlobject) turbogears component that exposes a REST api, ships with its own snazzy ajax tagging client, and can be used across frameworks, languages, and platforms. We are even hoping it can help make the world a better place.

So StickyTags (which doesn’t exist) and PloneTasty are two implementations of the same concept, with StickyTags being the AT/zodb implementation and PloneTasty/tasty the new microapp design (mashup architecture?) we have been working on and are pretty psyched about.

And if you act now, you get the knife set and the lint remover too, for just 3 easy installments.

Filed by jonah at 9:38 pm under plone
No Comments

Originally published on theploneblog.org

In this episode, Sean Kelly at NASA compares j2ee, rails, zope/plone, turbogears, django… cue the laughtrack

Okay, this is a long one, but it rivals any comparison matrix:

better-web-app

Plone comes out shining, although arguably it compares apples to pomellas.  Someone with the chops should really cut this baby up into chapters, cause its  a win for dynamic  languages over j2ee, and python, and Plone to boot. (spoiler: he uses the zmi for “hello world” and ArchGenXML for the time tracking app).

I could also connect this presentation to ideas in Johnny Can’t Program (cited in a recent interview with Behlendorf) and “where have all the zmi developers” gone, but  I’ll save that for another post.

Filed by jonah at 9:29 pm under plone
No Comments