Menu

Colin Devroe

Photographer. Podcaster. Blogger. Reverse Engineer.

Decentralizing all of my data

A few days ago I came across Ton Zijlstra’s post about trying out Obsidian. I didn’t have the time to read it just then so I quickly stored it in Unmark (shameless plug alert) to read later.

After reading his post I realized he is attracted to Obsidian for the same reasons that I was to give it a try. It stores your notes in a format that keeps the data local, readable, and accessible to any text editor. I really like Simplenote but it only stores notes in Simplenotes cloud service and not as local files (although, it did before Automattic acquired the service many years ago).

As the capabilities of cloud services get more and more robust I find myself drifting back to the early days of my computing where I wanted complete control of all of my data. Even though I can have all of my data accessible from any device, anywhere in the world, doesn’t mean that I want to rely on those services to provide me that data. And, I certainly don’t want the data to be locked into any single app.

Web 2.0 addressed this issue by pushing for accessible APIs that allowed data to be imported or exported in a variety of standard formats. Which is great! However, I find that it is the storage that should be standardized and not the method by which you can move data around. And text notes are just one example. I’ll give another example in a moment.

Zijlstra links to another one of his posts regarding “networked agency” that leads to a super interesting post by Ruben Verborgh titled Paradigm shifts for the decentralized Web.

I need to spend more time digesting Verborgh’s post (which I somehow missed when he published it late last year). However, the principles that are mentioned within it really land with me. One that stands out:

As apps become decoupled from data, they start acting as interchangeable views rather than the single gateway to that data.

This is exactly what I want for so much of my personal data. Any note taking app should be able to read notes written in any app – making the app’s interface simply a view into my note data. Like email. I can choose any email app for any email address. And switch whenever I want.

I’m currently working at building the very same capabilities for my photo library. In the past, I’ve written about why Photos for Mac isn’t a good long term storage solution for our photo libraries. As I write this I’m creating a set of tools and a workflow for my photo storage that allows me to store my photos in a readable format and structure while retaining the ability to use an app like Photos for Mac as a view into that library.

I still have more work to do, and I plan on writing a post sometime in September on what I’ve made so far, but it is a challenge to be sure. My overall goal is to have a photo library that does not require an app, but that an app enhances the experience of browsing my library. And, that all metadata related to a photo (EXIF, tags, faces, etc.) are stored within the file itself and not only within a photo management tool’s database. This will not be easy but I’m determined.

Notes and photos are not the only data that I wish I had in a standardized readable format. My links from Unmark are important to me. Unmark can already import from a variety of services and formats, and it can export into a JSON file. But I’ve recently started working on an export process for Unmark that will spit out a standardized HTML file so that people can export their links to be ingested in any browser or app they desire. This is, of course, the Web 2.0 approach.

I’m glad I read Zijlstra’s post. It has reinvigorated me to continue my effort to decentralize as much of my personal data as possible.

My Web: Yesterday and Today

This “Web 2.0” that we’re all so accustomed to lately is great. Semantic, accessible, open, and dripping with fantastic design. However, there are times I reminisce about the days of old, the days of well – Web 1.0.

There are several sites, some still in existence that I really do miss. I remember spending hours on the old Deviant Art just trying to find minimalist desktops and indy art. I also remember digging, refreshing (what is that anymore?) and bookmarking countless pages on The Force.net to find the latest and greatest information on release of the Star Wars Special Editions. I remember pulling my damned hair out trying to get ASP to do what I wanted by using Microsoft’s documentation.

It goes beyond sites though, since back then the web wasn’t about usage but rather building the foundation for what we have today. Using the Web in the 90s wasn’t, for me, about sharing photos and bookmarks, or creating and distributing content quickly and easily, it was about communication and expression of thoughts via hypertext. The more I think about Web 2.0, the clearer the picture becomes about the Web as a whole. We have an insanely far distance to travel before the Web becomes what it has the potential to be. Obviously services like Flickr, Del.icio.us, and Newsvine are getting closer to what we’d all love to see replacing Deviant Art, e-mailing bookmarks, and CNN, but they are still only very simple concepts done in fairly complex ways.

I listened to a few of the Carson Workshop podcasts and, I must say, I realized how complex our jobs sound to the “average uninformed developer”. Combine the complexity of learning the “best practices” in Web development with how many developers out there that are still using tables for layout, Microsoft Access databases, and reading Lockergnome for HTML tips, and you can see that we’re not even close to where we could/should be.

What makes it even worse is that the people that could be advocating these changes in the new and ignorant developers, are resting on their laurels or even bad-mouthing efforts to help out. Perhaps such efforts as Naked CSS Day won’t make a large impact on Web Standards Awareness, but who cares, at least Dunstan Diaz is trying to do something about it!

“Back in the day” (according to Dane Cook this was indeed a Wednesday) I was always amazed when new specs were released, new technologies developed, or different ways of accomplishing tasks were mastered. Nowadays, I see a lot of copying going on. Sure, we have our elite few that are definitely leading the innovation pack, but in the old days everyone was an innovator. If you couldn’t get something to work, you figured out a way to do it regardless. You busted down walls, you hacked like a mad-man, until finally the result you were looking for was accomplished. Nowadays, you run to Google and do about four searches and copy what someone else has done right from their site. Sometimes this is good, but if you find yourself doing this every time, it isn’t.

I suppose I miss the speed at which innovation seemed to be moving on the Web. Even at 56kbps and under, it surely seemed that the Web was changing faster in 1996-1999 than it is now. I think we’ve hit a Web 2.0 plateau where 10 major services were released and everyone else is trying to catch up with them instead of trying to do better than them.

Take microformats for example. A great effort. Standardize the way specific chunks of content are marked up, this way it will make it much easier to move, distribute, and work with going forward. However, some of these standards are just atrocious. I look forward to trying my hand at making some updates to some of those specs in the near future, but instead of trying to simply use microformats, we need more than just five people thinking about how to improve them.

Recently we’ve seen a gathering for an RSS Advisory Board. Thank heavens, the last guy that was running the show was not only an asshat, but he made Communism look like Kazaa (if you don’t get this joke you probably have a life, which is cool – can I borrow it?). I’m looking forward to seeing what happens with RSS in the near, and distant, future.

AJAX. Oh god, do not get me started. A superb effort has been put into improving not only awareness but accessibility, implementation, and documentation of the HTTPRequest Object. Sure, we’ve had these types of abilities for ages, but I still think all this “excitement” will lead to one good thing – improvements. Ajax, while not revolutionary at all, has caused many newbies to open their eyes to, not only standards (due to the use of XML, etc), but also to the thinking a little bit beyond the separation of presentation and content – but also of functionality. I’d like to put a name on this particular movement, but I doubt the World could hold such an acronym.

I said we’re on a plateau right now, but I think that might be incorrect. Rather, I believe we are on the escalator. The down escalator. And, instead of actually going down with it, we’re trudging onward and upward – each foot landing on the next step only to find another one approaching right after it. This battle to make the Web better may never really “end” but I definitely think we need to pick up the pace a little. Like back in the old days when we said “Screw you” to tables for layout, WYSIWYG editors that wrote horrible HTML, and oh yeah – Windows servers.

[tags]internet, web 2.0, ajax, web development, programming, microformats[/tags]