Alchemical Musings

Originally posted on theploneblog.org

Free Software Developers at Work and Play

I haven’t posted much here lately, but I have been writing. I recently
finished my first semester as a doctoral student in Columbia’s school of journalism and one of the papers I completed draws directly on my experiences in the Plone Community.  A few years ago I remember being struck at how different open source development was from what I (and presumably others) imagined it to be. I kept pitching human interest stories to journalists, ones that might emphasize the playfulness, the sprinting, and the organizational experimentation, but got very few nibbles. So, I finally wrote some of this up myself before it all fades from memory:

Fabricating Freedom: Free Software Developers at Work and Play

The paper was for a wonderful class this semester at the New School taught by Paolo Carpignano (The Political Economy of Media – here is the syllabus).
The class was all about the shifting relations between fabrication and
communication, or more colloquially, work and play. We opened with Marx
and Hannah Arendt and closed with Yochai Benkler and danah boyd. The piece I wrote is personal and anecdotal, but reflects on all that our community has taught me about free software, free culture, organizing, consensus building and the day to day politics of software development.

enjoy.

Filed by jonah at 10:16 pm under plone
No Comments

Originally published on theploneblog.org

Open source software as pedagogical scaffolding, and F/OSS ecologies as a dialogical knowledge communities.

This is a fun post recognizing the role of open source software and breaking routines in learning new programming patterns and paradigms.

7 Reasons I switched back to PHP after 2 years on Rails

Rails was an amazing teacher. I loved it’s “do exactly as I say”
paint-by-numbers framework that taught me some great guidelines.
I love Ruby for making me really understand OOP. God, Ruby is so beautiful. I love you, Ruby.
But the main reason that any programmer learning any new language
thinks the new language is SO much better than the old one is because
he’s a better programmer now!

This story articulated an idea that I have been thinking about for a while – the ways in which developers working on open source software enter into an educational relationship with the software and the community (sometimes indirectly, mediated through the code).

I have often appreciated all that Plone has taught me about the domain of content management, component architectures, extensible software design, internationalization, test driven development, responsible release management, etc etc. I know that it has taught me well since when I walk up to new pieces of complex software like Sakai or Drupal the concepts are familiar and often isomorphic. I can vouch that exposure to Plone has helped designers I know with stretch their CSS skills, improve the accessibility of their sites, and more cleanly separate presentation from logic.

I have also made the case that for an organization to work on software in isolation is bit like having a conversation with yourself. At first you might arrive at some new insights, but its really hard to learn anything new in a hermetically-sealed vacuum chamber. The software world transforms so quickly that it is a Sisyphean task for any one person or organization to track. Joining a community, even if it is through the indirect, intermediary object of code, is a great way to continue learning and stay on top of emerging trends. The notion that learning happens through dialog is an old one; the notion that working with open source software is a form a dialog with the authors is a bit less obvious.

My main critique of Derek’s post is that he doesn’t explicitly acknowledge the fact that Rails is open source, which is precisely what enabled him to learn so much from the framework. This isn’t just a matter of attribution, it has practical implications for being able to continue learning new tricks over time.  If he had realized that Rails-the-software embodied the knowledge of the Rails-the-community, he might not have been so quick to venture off and write his very own framework. I am not arguing that he should have steered clear of php, but he does not explain why he decided to roll his own framework as opposed to joining forces with Cake or another existing php framework, or at least using an existing php templating system. With a more explicit understanding of the origins of the knowledge that Rails-the-software captures, he may have appreciated the potential future gains of remaining part of some community.

While it is possible to learn something from proprietary frameworks, “learning” is fundamentally about the open exchange of knowledge and meaning, which are values intrinsic to F/OSS. While you can learn something from a .NET api, you can’t step through the entire stack of software in the debugger. Perhaps more importantly, the cultural tendencies on an open project support constructionist poking and prodding (dare I say, hacking?).

Filed by jonah at 9:21 pm under earth,epistemology,plone
No Comments

Originally published on theploneblog.org

Beyond the CMS – What are Plone’s greatest future competitors?

I recently encountered O’Reilly’s asymmetrical competition meme and think its a good jumping off point to discuss the differences between Plone’s perceived and actual competition.

First, let’s catch up to where we are today:

Web 2.0 … The Machine is Us/ing Us

The opensource CMS horserace has seemingly settled on a few players, and without provoking any religious wars, I continue to be impressed with the richness and maturity of all of these projects.

But here in the educational sector there are rumblings which I think will spread beyond our corner. In our world ‘C’ stands for Course, not ‘Content’, and the big players are Blackboard (which swallowed WebCT), Sakai and Moodle. Here too, competition may come from surprising corners, as the game itself changes beneath us.

“Collaboration via the net does not necessarily require monolithic, expensive tool suites that aim to do everything under one umbrella. We will share and demonstrate the use of readily available, mostly free, discrete sets of “small” and “loosely joined” technologies – weblogs, wikis, instant messaging, audio/video chat. The loose joining means that how they are connected are not necessarily in the programming of the software, but the ways people can use them in a social context that is an environment of dynamic, changing relationships and connections, rather than the rigid, limited ones defined by computer code.”

from Social software: E-learning beyond learning management systems.

This argument is elaborated on, with many examples of applications that might work this way in this paper:

EMERGING TECHNOLOGIES Tag Clouds in the Blogosphere: Electronic Literacy and Social Networking

So what does this mean for the other sectors where Plone operates? I have been hanging out in Drupal land lately, and for a variety of reasons I don’t consider that platform to be a serious threat to Plone, in the long term (more on that in another post).

On the other hand, and this might raise some eyebrows, folks maybe should take a peek at Gallery 2.2 . Yeah, its in php, and they don’t have enough unit tests, but it does a really nice job of solving the “bucket” problem – that is, easily publishing a repository of digital assets on the web. Their next version will handle audio and video files, and the software is popular enough that people have built desktop clients for iPhoto and Picassa. They have a good story for multi-site installations, upgrades, and even a web based mechanism for upgrading plugins. Just imagine mashing up this backend store with a social-networking tool like elgg. Maybe you could create complex and elaborate views of your data with widgets coming out of the simile project (exhibit and timeline, in particular). You might even be able to use a visual programming tool, like Yahoo’s pipes or IBM’s QEDWiki to assemble this application.

The real threat here isn’t Gallery. Its the loosely-joined, disconnected applications that are becoming connected through the component architecture of HTTP itself (plus a few decent patterns and standards). No one wants to be trapped in a silo, not even if its decked out with hardwood floors, leather furniture, and a marble mantle.

Plone can be a major provider in this hub of communications, maybe even sometimes at the center. But we do need to try to anticipate the future role of the CMS in the face of asymmetric competition.

As Laura Trippi once put it, Content Management Systems like Plone,
are turning content produces into coders, and vice versa. We’re
creating monsters, and they might soon be outgrowing the tools they
were weaned on.

(thanks to Biella for the video reference and the critical commentary).

Filed by jonah at 10:06 pm under plone
No Comments

Originally publihsed on theploneblog.org

How hybrid economies help keep software honest.

Last week’s Plone Conference was truly phenomenal – provocative, intense, and fun (big thanks Jon and ONE/Northwest!).

One of the most amazing things I experienced last week was alluded to in Eben Moglen’s keynote (to be posted soon)- the manner in which this community has managed to bring together people who don’t ordinarily interact.

Throughout the breakout sessions, I continued to question dividing us up according to our respective vertical sectors – Corporate, Non-Profit, Educational, and Government. As I have begun to write about elsewhere, systems like Plone can help balance the flow of communication and power between people in a variety of situations and settings. Content, collaboration, and community are contexts which exist across sectors, and the tools we all need cross over as well (sometimes with slightly different tunings).

In many ways lumping together all the folks involved with education is odd. Universities are microcosms of cities, and their IT needs are as diverse as the the rest of the world. However, there are still structural and social similarities that form the basis for common language and culture. After engaging with my fellow educators a the educational panel session and the BOF session I understood the value of us sharing and strategizing, beyond just commiseration.

But through it all, there was one thing that united all of the different attendees – a piece of general purpose software called ‘Plone’.

It is worth dwelling on this mixture of participants and the varying forces they apply to the software. Lessig and Benkler have both been writing a great deal about hybrid economies lately, trying to understand their rhythms, and how we might be able to design them to succeed. They have been writing generally about the “commercial economy” and the “second economy” (sharing, social production, etc), but the lessons may cross over directly to our community.

I realized in Seattle how beneficial diversity can be for software production.
Most of the consultants using Plone are there strictly for traditional market considerations – to make a profit. They are helping to keep the software honest. Unlike some other open source projects which exclusively service the educational world, Plone is not sheltered from the raw, harsh forces of the commercial market. This means that some of the people using Plone use it because it helps them get their jobs done efficiently. Others have called this “productivity arbitrage“, and it is a concept that may hold the key to designing successful open source projects.

It is challenging to imagine working backwards and trying to design a software ecology which captures the hearts and minds of such a diverse following. No small task.

As Rheingold said “There’s been an
assumption that since communism failed, capitalism is triumphant,
therefore humans have stopped evolving new systems for economic
production.” – Is Plone’s ecology an example of one of these new systems, and if so, what are our distinguishing characteristics?

Filed by jonah at 10:13 pm under plone
No Comments

Originally published on theploneblog.org

The $100 laptop project has chosen Python as the primary development language for The Laptop.

I was lucky enough to get my hands on an olpc developer board, and have spent a little time learning about the platform and project.

While there are a few issues I have with the project, it is really an thrilling moment in educational technology and after holding the hardware in my own hands I actually believe this vision might truly manifest.

The main reason I am writing about this in the Plone blog is I have learned that the olpc’s application development language of choice is Python!

Sheckitout.

While Plone itself is probably not well suited for the laptop itself — The Laptop’s hardware characteristics are closer to a pda w/ a big screen than a MacBook Pro  (plone-on-a-stick? maybe it could ship on a thumbnail drive), it is easy to imagine communities of practice emerging around this platform. Places where educators and students alike can share tips and tricks, strategies and pitfalls. Who will be their dotmac?

Admittedly, these devices are being built to operate unconnected to the Internet, communicating with each other through ad-hoc mesh networks (presence will be a very low level primitive in this environment, and all applications will have access to it), there may still be a role for a server w/in the network.

I don’t know exactly how Plone fits into the larger OLPC strategy, but I get the sense that with all the momentum and capital around this project, if Plone gets there lots of people will see it. And many of them may be the next generation of Python/Plone developers.

Filed by jonah at 10:04 pm under plone
No Comments

Originally published on theploneblog.org

Preliminary report on the Big Apple Sprint

July fourth has come and gone, but the fireworks set off at last week’s sprint are still visible.

The sprinters arrived at Columbia University bright and early, Wednesday morning. (note to all future sprint organizers: tell the caterers to skip the decaf and double the regular order). About ~13-15 sprinters were present, but we also coordinated remote sprints with Austria (+5 hours ahead) and Utah (-2 hours behind) meaning we were basically sprinting around the clock.

We all used the freely available, plone-based, OpenPlans service to manage our collaboration and everyone found the software to be extremely reliable and easy to use.

The sprint began with introductions and detailed demos of the tools and
products people had been working on and were most proud of. Sprints are
difficult to plan in advance since the skills and interests of the
attendees are not decided until the final roster shows up. A diverse
range of interests were represented, but common themes rapidly emerged
– most of us were working at or with non-profits, building community sites with educational goals,
and we all had a very strong interest in participatory media (in contradistinction to mass broadcast media). It became
apparent that although we were all working on different problems, very similiar features and tools could be part of our solutions.

The sprinters self-organized into 3 teams (+ 1 remote team):

Multimedia – focusing on improving the handling of multimedia
content w/in Plone. Topics included transparent management of multiple
media formats, improving the quicktime player, abstracting the common controls from the different media player formats, merging Austria’s
ATVideo bittorrent branch, allowing for remote resources to be managed by the
media types, and the integration of CCNMTL’s video clipping tool into
PloneMultimedia. Thanks to Nate, Gary, Anna, Kurt, and Sky for making this group a productive success.

Discussions emerged around the hybridization of modern media formats.
Is an audio track with synchronized gifs a piece of audio or video media? Modern events are now distributed in multiple media
formats (e.g. podcasts and vodcasts) – might PloneMultimedia benefit from a
UI analogous to PloneLingua? What about metadata embedded directly in
the binary file? Should it be editable ttw? Which file is the
canonical file and which are the derivatives? Whatsoever will we all do about large file support?!?

A new ‘media’ container was introduced to PloneMultimedia allow for the mgmt of media that spans multiple traditional formats.

Annotations/Tagging – laying out the jigsaw puzzle that tagging,
rdf, taxonomies, folksonomies, and sticky notes, and microapps have become in the hopes of
consolidating on a common strategy to move forward. The Yucca project was born
after we all began to realize how many of our problems would naturally fall into place with a robust engine which supports user contributed content annotations.

Also, work was done by Anders and Chad on the sticky notes product with the aim of
factoring out the notes so they could be used outside of plone too
(with the persistence abstracted, so that it could be backed by a
microapp – like pita, or even stored client side), as well as improving the
editability of the notes – they now support “double-click to edit”.
Great job!

Blogging/Syndication – This group (Rob and Kurt) was primarily working on polishing quills so that it provides a smooth user experience.
Progress continues and Quills is looking like a serious contender.

Content Licensing – see Nate’s post on conetent licensing in plone. This work was conducted primiarly by the group working in Utah, out of C()SL.

In case its not obvious, there was a great deal of overlap between the interests of the groups. For example, the multimedia team was also very interested in tagging, syndication ((p/v)odcasts), and licensing. Although it initially seemed challenging to tease the participants apart, the groups self organized quite organically.

Some time early in the sprint, Rob treated us all to an impromptu tutorial on z3 annotations and continued to support the various groups in their attemps to absorb the new world order. By the end of the day, 3 seperate pairs of developers had the contentratings product installed and running, demonstrating how important it is for there to be good examples representing best practices in the collective – these practices spread virally. Whit actually spent a chunk of time with Christof and Darian thinking about a good packaging system for keeping the z2 and z3 layers as thin as possible. They also spent time backporting the tagger product from zope 2.10 to  2.9.

Throughout the sprint curious academics wandered through in an attempt to catch the rare glimpse of a geek working in the wild.

Ian Bicking also stopped by to check in with his new coworkers.

Since 3 days of sprinting is never enough, sprinting actually continued over the weekend at the openplans offices with a reduced number of hardcore plonistas. Christoff was gracious enough to invite us into his home for a home cooked meal.

If anyone has anything else to add to this report, please do – sorry I missed it.

The sprint was very productive, educational, and great fun as well.
Beyond the technical achievements, relationships were forged that we
expect to flourish in the months to come. I think we all witnessed
tremendous convergence across our organizational requirements, and are
also convinced that the tools we are working on will be in great demand
once the corporate world figures out how useful these technologies can
be.

Throughout the sprint participants shared tips and tricks,
demonstrating how important live, in-person communication remains, even
in the Google Age. This was especially apparent when the nyc sprinters
attempted to convey proceedings to the remote sprinters and we felt the
limitations of geographic displacement.

Sky got married on Sunday in a park on the Hudson river. His last days as a
bachelor were devoted to this Sprint, and we all wish him and his new
wife a happy and healthy (Plone-free) honeymoon.

Pictures

“bigapplesprint” on flickr.com

And, since Anders is so special, here is his set:

Special thanks to our hosts, Columbia’s Center for New Media Teaching and Learning for sponsoring the sprint and making us all feel at home in New York City.

Filed by jonah at 10:01 pm under plone
No Comments

Originally published on theploneblog.org

A forray into drupal 4.7’s taxonomy system and what Plone can learn from it.

I have been moonlighting on a Drupal project and paying close attention to their taxonomy system. Drupal’s taxonomy/category/tagging system was completely revamped for their 4.6->4.7 release – a release close to a full year in the making, analogous to the Plone 2.0->2.1 “minor” point release.

The site I have been working on, theicarusproject.net

has a very rich collection of content, and one of the primary motivations for the migration is to get a better handle on the classification system – noboday can find anything on the current site. They were committed to Drupal long before I arrived, so I dug in with the hope of learning something from the contrast.

PHP bashing aside, there are alot of interesting things happening in Drupal land. I hope to follow up this post with a few more cross-pollinating nuggets, but for now I will focus on their taxonomy system.

Taxonomies in Drupal are considered the heart of the system, and the essential modules ship with the core and cannot be disabled. Most URLs in Drupal are effectively queries, much like our smart folders (actually, for anything aside from anti-chronological display order you need to install the Views module) but the display results are all instances of content with matching vocabulary terms. The absence of folders and containment initially confuses many administrators, and renders breadcrumbs largely useless, but does allow for the creation of sophisticated information architectures.

Taxonomies are managed top-down, not bottom-up, and have a separate administrative interface for their creation and management. Once the taxonomy vocabularies are created, specific terms can be added to these vocabularies without having to create content associated with those terms (in contrast to a bottom-up category system, like the mediawiki).

Category Management – Vocabulary Listings:

Druapl supports multiple vocabularies, which can each be associated with one or more content types. Vocabularies can be flat, one level deep, or N-levels deep (hierarchical). They can be fixed or free form (meaning content authors can make up new categories upon content creation). The core tagging system does not support the creation of tags per-user, per-object – only per-object.

Category Management – Add a Vocab:

Category Management – Add/Edit a Term:

The Drupal taxonomy system is very powerful, but its power is very open ended and does not necessarily lead users towards a uniform experience. The confusion around categories and taxonomies is best exemplified by the category module meant to consolidate and simplify taxonomy and navigation, but there is no consensus on its incorporation into the core.

A large number of modules are built around taxonomies. Core Drupal supports roles, but no groups (organic groups is a popular access delegatoin solution, but it is incompatible with other access restriction modules – so you have to choose one), and does not have a notion of containment (ie folders). So, for example, one way to restrict editing access is by enabling the taxonomy access module. Another useful module is the taxonomy browser which allows for advanced search against unions/intersections of vocab terms.

Category Browser:

Once vocabularies are created, and terms added, content can be associated with these terms:

Working on this site really drove home the value in separating the navigation axis (section) from the thematic axis (keywords), and separating these dimensions was easy to accomplish with the taxonomy/category tools built into drupal. In particular, once the scheme was developed, managing vocabulary lists (even hierarchical ones) is intuitive, albeit slightly clunky. I further chose to introduce a free-form tagging dimension for member contributed posts which may or may not fit into the fixed taxonomy. This is similar to myspace and facebook allowing for free-form hobbies and interests, and banking on a large enough user base that there will be overlap and potentially interesting intersections.

The system still does not allow for the intuitive modeling of a many-to-many relationship, which I continue to think is the litmus test which will mark of a truly powerful taxonomy UI. There is still quite a bit of programmer know how involved in setting up this system so that it operates the way that content administrators expect, and arguably there are too many degrees of freedom introduced by such a general purpose modeling capability (if you think about it, a tagging system can essentially allow web administrators to model relationships which used to require programming custom applications against an rdbms).

Nonetheless, Drupal’s taxonomy/category/vocabulary system definitely captures a few use cases more elegantly than Plone’s current core does. But perhaps the real lesson is the importance of not mixing navigation space and content space, which can be kept separate in Plone, but is all too easy to conflate (in Drupal too!).

Note: most things I describe in this case study could have been accomplished within core Plone – I think the most interesting things here are the administrative UI for multiple vocabulary management, the different types of vocabularies, and how central they are in the construction of a Drupal site.

Filed by jonah at 9:41 pm under plone
No Comments

Originally published on theploneblog.org

Do not try to bend the folder — that’s impossible. Instead, only try to realize the truth. Then you will see that it is not the folder that bends–it is only yourself.

Tagging seems to have spurred a growing amount of research on categories and classification.

A recent paper by Clay Shirky, entitled Ontology is Overrated: Categories, Links, and Tags directly challenges the desktop metaphor which currently underlies much of Plone’s UI. To be sure, it is certainly possible to model the connections that Shirky describes using topics, smart folders, and a disciplined use of keywords, but the metaphor is critical for designing and intuitive system, all the way down to the icon.

I have recently been working a bit with Drupal, and their handling of this problem is worth checking out.  The core taxonomy module, combined with its corresponding menuing systems (menu, sitemenu, taxonomy menu) provide a great deal of flexibility in this regard.

Organization is going organic.

Filed by jonah at 9:34 pm under plone
No Comments

isomorphic surprises: stickies, tasty, and the importance of user contributed content

I have been thinking alot about tagging lately, especially how a complete tagging system – comprised of user-item-tag triplets, is isomorphic to rdf‘s subject-predicate-object triplets. It is amusing to think about how egocentric Web 2.0 is – The subject is always me. Web 2.0 might be made of people, but not just any people – Just the most important one in the world.

Today I had a fun time trying to explain to people on irc the power and importance of user contributed content annotations within plone. Crucially, user content annotations are per-user, per-object, and in many cases a single user might want to annotate a particular object with more than one annotation.

Interesting annotations can come in many flavors. There are free form notes, fine grained annotations (anchored to particular phrases – think msword trackback – or geometrical coordinates on the target – nice for image annotations), keyword annotations (aka – tags), etc ect. There many problems that can be solved with custom per user content annotations, including quiz and poll results, per student answer submission, and lately we have been working on allowing users to clip audio and video by annotating start and end times on media objects.

It is important not to confuse per-user tagging with DC:Subject – the dc metadata is shared across all users (like categories in the wikipedia) and in that sense, is global. While we are on the topic of tags, it is useful to talk about the vocabulary that drives the tagging. Vocabularies can be fixed or free, individual or collaborative, and personal or shared. All of these variations are interesting in different cntexts, and have to do with whether or not I see your tags, or if we each are developing our own ontologies.

In educational technology annotations are a big part of the problems we are trying to solve, but there are tons of use cases in the world at large. Additionally, a high performance, robust tagging engine can power personal content organization, like gmail’s labels.

Which brings me to the products we have been developing at CCNMTL. We have been using PloneStickies, a general purpose content annotation framework, in production for over a year now. Built using AT References, it allows us to create per-user AT objects connected to the target object. AT Schema Annotations won’t do the trick here since, like DC:Subject, these annotations are instance-wide. Z3 annotations might work, but by building AT derived stickies, we pick up search, workflow, permissions, and the richness of AT. This allows us to quickly and easily develop custom stickies, like the StickyClip.

PloneStickies has a ways to go, but the basics work great. It is not yet super useful out of the box, since the portlet it ships with only allows users to attach a single free form StickyNote to the target object. But it is great to develop applications with. It ships with with some super snazzy css stickies, complete with colored/resizable/title-barred/drop-shadowed/roll-upable/transparent-when-dragging notes, which can preserve their own x-y position and state across sessions and never fall off your screen. It now supports attaching multiple stickies to a target, but does not yet provide a mechanism for the target object to place the stickies itself.

At first we thought we could implement a Plone tagging solution using this framework – just create a StickyTags made out of keyword fields, and voilà – plonr. Trouble was, since tagging is such a symmetrical model, its tough to build an efficient zodb implementation (for me, at least) that can handle all the querrying we wanted to do.

Enter the tasty microapp and the PloneTasty proxy (about 90% done). Tasty is a stateful (sqlobject) turbogears component that exposes a REST api, ships with its own snazzy ajax tagging client, and can be used across frameworks, languages, and platforms. We are even hoping it can help make the world a better place.

So StickyTags (which doesn’t exist) and PloneTasty are two implementations of the same concept, with StickyTags being the AT/zodb implementation and PloneTasty/tasty the new microapp design (mashup architecture?) we have been working on and are pretty psyched about.

And if you act now, you get the knife set and the lint remover too, for just 3 easy installments.

Filed by jonah at 9:38 pm under plone
No Comments

Originally published on theploneblog.org

In this episode, Sean Kelly at NASA compares j2ee, rails, zope/plone, turbogears, django… cue the laughtrack

Okay, this is a long one, but it rivals any comparison matrix:

better-web-app

Plone comes out shining, although arguably it compares apples to pomellas.  Someone with the chops should really cut this baby up into chapters, cause its  a win for dynamic  languages over j2ee, and python, and Plone to boot. (spoiler: he uses the zmi for “hello world” and ArchGenXML for the time tracking app).

I could also connect this presentation to ideas in Johnny Can’t Program (cited in a recent interview with Behlendorf) and “where have all the zmi developers” gone, but  I’ll save that for another post.

Filed by jonah at 9:29 pm under plone
No Comments