Menu

Colin Devroe

Reverse Engineer. Blogger.

Following Twitter accounts via RSS

I haven’t missed Twitter that much since deleting my account. The first week or two I missed Moments – but once that subsided I realized that Moments are generally a waste of time. Realtime reporting of most newsworthy events result in ill-informed, unsubstantiated tweets. I’m at a point now where I’d much prefer to get the real story after-the-fact rather than realtime.

There are instances where realtime reporting can be incredibly useful, such as when there is a fire, a traffic accident, or a natural disaster happening. Those tweets can save lives. But, in general, I’m perfectly OK with reading up on the news once or twice daily to see what really happened.

I do miss certain Twitter accounts. Especially those that do not have a blog or web site counterpart that I can follow along through another medium. And since Twitter is still web and developer hostile (meaning their API is far too limited and they don’t support open web distribution technologies like RSS) I’ve missed out on a lot of great content from those Twitter accounts.

So today I went searching around for some RSS feed generators that would use what little access to Twitter they have (presumably the limited API or HTML scraping or both) to create an RSS feed from accounts or hashtags or lists. And there are a number of services out there, some of which you have to pay for, others that toss in some ads, or others that are severely limited.

Then I found Publicate. I’m using Publicate’s Twitter RSS Feed Generator to create a few feeds based on some Twitter accounts I miss the most. You simply type in the URL you want to create a feed from, give them your email address*, and they provide a feed URL. So far it seems to be working. I’ve created a new collection in Feedly to store these feeds. Hopefully I’ll get the tweets I wanted to see most and I won’t have to deal with the drivel and hate I’ve seen on Twitter over the last 18 months. Or even Twitter itself!

* I certainly don’t mind my email address being a form of payment to a company. So I gave it to them. But, if you’re a bit of a hacker it is quite easy to dismiss the overlay, read the page’s source, and grab the feed URL without giving Publicate your email address. I want this tool to stick around so if my email address helps them to keep it up-and-running so be it.

August 8, 2017

JSON Feed to Twitter using PHP

In 2009 I scrawled together a simple PHP script that tweeted links based on an RSS feed. I then updated it to support OAuth and open sourced it on GitHub.

I haven’t really touched it since (though I get about 3 emails a month about it). Just a small update here and there.

This morning, with all of the JSON Feed hubbub going on, I decided to recreate that same simple script to parse JSON Feed rather than RSS. I also updated to the latest release of the PHP Twitter OAuth class by Tijs Verkoyen.

You can download the latest release or clone the repository on GitHub.

This took me mere minutes thanks to JSON Feed being much easier to deal with.

June 2, 2017

Avoid being great at Twitter

Seth Godin:

You can be good at Twitter in about five minutes a day. Spending ten minutes doesn’t make you twice as good… in fact, there’s probably little measurable improvement. To be great at Twitter might take five hours of daily effort.

At over 48,000 tweets I do not need any more Twitter practice. Lately I’ve been spending just 30 minutes now-and-then to go through my Twitter Lists and retweet that which I find positive, interesting, creative. Twitter has become such a mire of hate and political bunk that I simply avoid it otherwise. So while Seth’s advice is to save time by being happy with being good at Twitter as opposed to being great at Twitter I’m limiting my exposure to it simply for my own well-being.

Side note: the above may lead some to believe I no longer find value in Twitter. I do. Twitter Moments and Trends and Searches are still incredibly valuable. I simply find no value in keeping Twitter open as I used to. Partially due to the fallout from the US political core but also from Twitter’s own product decisions to algorithmically castrate its timeline.

April 18, 2017

The slow web and POSSE

David Mead:

This year all of my posts, replies, and retweets on Twitter will be coming from this blog and not using the Twitter app (#OwnYourData). That probably means doing it at the end of the day. I’m hoping that will make them more considered (something we may all want to be in the coming years).

I have most notifications off (and have for years). And I plan on keeping it that way.

But, I’m not doing so well on what he’s talking about in the quoted bit above. POSSE, as the indiewebbers call it, is posting on my site here and then syndicating it elsewhere. My blog posts are syndicated to Twitter the way I’d like but not Facebook or Instagram (the other two networks I use the most). And I also find myself lazily posting directly to Twitter rather than through my site because the apps are so easy to use. I wish I did better.

Here is what I would need to do to pull this off personally:

  • Post status updates, posts, audio bits, and photos to Facebook
  • Post photos to Instagram
  • Be able to retweet or quote tweet posts easily from my site (no idea how to do this)
  • Show Twitter likes, replies, retweets, quote tweets on my site
  • Show Facebook likes, replies, shares on my site
  • Show Instagram commends and likes on my site

I wouldn’t have to do all of these to be happy, but I’d at least like to push the content to those networks. Maybe I’ll start there.

January 23, 2017

App.net shutting down

Dalton Caldwell:

We envisioned a pool of differentiated, fast-growing third-party applications would sustain the numbers needed to make the business work. Our initial developer adoption exceeded expectations, but that initial excitement didn’t ultimately translate into a big enough pool of customers for those developers.

I’ve been a paying subscriber to App.net for the entire life of the platform (that is, until they cancelled my subscription this week).

When App.net launched many were drawing a line of comparison between it and Twitter. And since this announcement I’m seeing many drawing a line of comparison between App.net and Micro.blog. This isn’t an apples-to-apples comparison.

If you read Dalton’s vision above, it doesn’t read anything at all like Twitter or Micro.blog.

App.net was an API for application developers to build on top of. Yes, something Twitter-like could be build on top of it. But so could so many other things. It had a data storage service, a push notification service, and even a crowd-funding feature called Backer that would, presumably, allow developers to pre-charge for new features for apps.

App.net was a very ambitious platform that, I believe, got pigeon-holed into a Twitter comparison because they created Alpha – a Twitter-like microblogging platform – as a demonstration of their own API. I think this muddied their messaging to the point where most people would describe App.net as a Twitter alternative.

Manton Reece’s forthcoming Micro.blog is not anything at all like App.net. Though, many are confused about Micro.blog similar to how many were confused about App.net. (I’ve had at least three conversations about Micro.blog where people have no idea what it will do.) They are comparing it to Twitter even though Manton doesn’t usually draw that line himself. And I think he will have to find a way to communicate its decentralization and the fact that it will work with your existing blogging platform so it too doesn’t get packaged and framed as simply a Twitter replacement.

January 14, 2017

Manton Reece on Twitter at 10 years

Manton Reece, like me, recently passed his 10 year anniversary on Twitter. I like this point he makes about how to see Twitter going forward:

It used to be impossible to imagine that Twitter could fail. And today, it’s still unlikely to vanish or even change much overnight. But the web will be better if we assume that Twitter is a lost cause. From the 10-year view, it’s clear that Twitter has already changed.

Assume Twitter (or any other service) could disappear and make adjustments where needed. I like it.

December 9, 2016

E15: Bots, Windows 10 Surface Book review, Twitter Head of Product

Last weekend Danny and I sat down and discussed our current experience with bots, the progress I’ve made on my still forthcoming Windows 10 and Surface Book review and also Twitter’s new Head of Product hire.

Links:

Download MP3

December 8, 2016

The Twitter grabbag

When I signed up to Twitter over 10 years ago the site was remarkably simple and easy to use. Back then using Twitter on mobile meant crafting SMS messages using particular syntaxes like “follow username” or “d username message”. It was a fantastically simple experience with huge implications on how we all communicate.

Today, there is no such thing as a unified Twitter experience. And all of the experiences are far from simple. Twitter on the web, or Twitter.com, feels pretty much the same across many desktop platforms. The issue I have with the web experience is that it is a pretty terrible way to use Twitter. The algorithmic main feed, the UI being shoved full of useless information, the confusing conversation layouts, all of these make Twitter.com not very fun to use. And the web experience on mobile is such a departure from the web site that I’d be surprised if anyone at all could use it. I’ve tried. A lot. And I wouldn’t wish it on anyone.

On the Mac there are several excellent Twitter clients to get the most of our Twitter. I remember when Twitterrific debuted. It was amazing to have a running feed to bring all of my online friends right into my office. Today, on iOS, the Twitter applications are second-to-none. I’ve used nearly all Twitter clients for each platform and iOS is, hands down, the best way to experience Twitter both via the official apps and the best client of them all; Tweetbot.

Tweetbot is so far and away the best way to use Twitter it isn’t even funny. If you like Twitter at all and use Tweetbot  you know how much value it brings you. You’re able to keep up with what is going on, easily create and reply to tweets, and search for what people are saying about a given topic.

On Windows, which I’ve switched to using this weekend, the Twitter experience is so bad that I have no trouble figuring out why Twitter has a hard time growing. A few weeks ago I quipped that if anyone was forced to only use the official Twitter apps or web site that they’d likely not use Twitter at all. The same could definitely be said for people using Twitter on Windows 10.

When I’m in a group of people and I’m the only person that really uses Twitter – uses it every single day – I always used to wonder why. Why aren’t these people on Twitter? Why aren’t they getting as much value from it as I have? I don’t wonder that any longer. The experience is terrible. For anyone using Windows 10 or Android I’d bet they open Twitter from time-to-time (if they have an account at all) and then close it wondering why people use it at all.

Take this scenario as a for instance. Someone signs up to Twitter, they follow a few accounts like a celebrity or two, their favorite sports teams, a journalist or two, a politician or three, and maybe one or two friends that are using Twitter. All told they are at around 40 accounts. And I’d say this is likely the median for Twitter. They log into Twitter. The algorithm shows them 4 tweets they “missed while away”. And the rest of their main feed is a hodge podge of some recent tweets. They scroll down once or twice but not much else is interesting and they move on with their day.

If this was how I used Twitter then I could never get any value from it.

Contrast that with the Tweetbot experience. I open Tweetbot and I’m presented with my main feed from the point I left off from. I have 56 tweets that I missed since the last time I opened the app. And, just like my email, I quickly glance at each one – sometimes stopping to follow a link or look at a photo – before moving on. I won’t be “done” looking at Twitter until I’ve gotten through all of the tweets. Like being done with looking at my email. For some people this may seem overwhelming. They may ask “how can I keep up?” They hate having emails in their Inbox and so they hate having Unread Tweets. But for many this would produce a valuable use of Twitter. The more tweets people see the more chance they have of seeing something that interests them. Of course that is the purpose of Twitter’s main feed algorithm, but more is more in this context. Ask any user of Tweetbot and they will tell you that they get value from being on Twitter. Ask any user of the Twitter web site and I’d bet they only see 20% of the content shared on the site and could live without the service.

To make matters worse, using Twitter on different platforms results in having different features. On some platforms there are Moments. And Lists. On others, they simply aren’t there. Or they are so buried you would never use them. Some have video. Some do not. Some have GIFs. Some do not. And many of the experiences do not support Polls. And so you see tweets that seem out-of-place. On some clients there are keyboard shortcuts. On Windows 10, in the official app, there are none. This is baffling. These little inconsistencies means that different sets of users have wholly different experiences in using the platform. How in the world could Twitter craft that experience to make Twitter valuable to everyone that uses it when they have no idea what features are popular across all platforms? It is a mess. Honestly, I can’t think of a single service that is as popular as Twitter and is as much a mess. Pinterest, Facebook, Skype, Snapchat. All of these are fairly consistent across all platforms.

I still believe Twitter is on the ropes. I’m sad but I believe they have very little chance of survival unless something big happens. And my switching to Windows and realizing that I’d rather not use Twitter on Windows 10 (and, the recent exodus caused by the political fallout) really makes me think that Twitter’s end is closer than ever before.

November 16, 2016

Tweeting for 10 years

Last week Jeremy Keith reminded me, yet again, of an anniversary I share with him. That is, we’ve now both been tweeting for 10 years. Here is my first tweet.

Jeremy beat me by 6 days and only 5,000 tweets. Can you believe that back then only 5,000 tweets were sent in 6 days? These days I’d guess that 5,000 tweets happen a few thousand times per second. And tomorrow, on Election Day, you can guarantee millions of tweets per second.

Jeremy reflects on the early days and also on some of the things that changed over time. Please, please go read his post. But I’ll expound slightly on what he’s written.

Most notably this bit:

The most obvious sign of change was the way that Twitter started treating third-party developers. Where they previously used to encourage and even promote third-party apps, the company began to crack down on anything that didn’t originate from Twitter itself. That change reflected the results of an internal struggle between the people at Twitter who wanted it to become an open protocol (like email), and those who wanted it to become a media company (like Yahoo). The media camp won.

If you listened to audio bit E8, wherein Danny and I chat about Twitter, one of my suggestions for Twitter is to go back to this. To go back to supporting third-party development. We chatted about the whacky uses of Twitter (like drawbridges, plants that need watering, etc.) but there are very, very practical uses too.

But now, just a few weeks later, I do not feel that would be enough to save Twitter. And I do mean save it. It is dying. It will go away. I do not see anyone coming in to rescue it at this point. In fact, if someone does step up to the plate to try to rescue it, it may be the wrong entity to do so and it may get worse.

Jeremy has a leg up on me that I do not have. He posts his “tweets” first at his site and syndicates to Twitter. Well, I do too. However, I don’t only post to my site. I tweet. A lot. It is a hard habit for me to break. I love tweeting during sporting events. I love even more tweeting during tech events like Apple’s Media and WWDC events or Microsoft’s Build events or rocket launches. In context they are fun, sometimes funny, sometimes informative to follow those conversations happening on Twitter. If I published those particular notes to my site first they’d be in a silo of sorts and out of context. Someone stumbling upon them would have no idea what I was talking about. So do I just not write those tweets any more?

Unlike Jeremy I will be sad if Twitter goes away. It has been part of my life for 10 years and I think it is the best social network we have going. But, like Jeremy, I’ll keep posting here. Because my site will be around for as long as possible.

Now I just need to break the habit of posting tweets to Twitter.

November 7, 2016

E8: Tesla, Twitter, Blogging

Extra special, and most likely reoccurring, guest Danny Nicolas (@djloche) and I have a conversation about Tesla, Twitter, Blogging and a bunch of other things.

Download MP3

October 8, 2016

Three microphones

I began posting to my own site in earnest on March 6th of this year. I wrote:

So, starting tonight that is what I’m going to try again to do with a goal of sticking with it in perpetuity. This doesn’t mean that I won’t be posting to Twitter or Facebook or Instagram, but that everything that I post there will originate here on my site. I may still craft those messages manually (since each network is so nuanced) but like Jeremy and Manton I will have to figure that out as I go too.

More or less this is what has happened over the last three months. It has been fantastic.

I’ve redesigned my site, added a few new post formats (more on that here), and have published a slew of status updates, photos, blog posts, and even a few audio bits — which I hope to do more of. I’ve switched platforms, tweaked my settings to no end, and tried a menagerie of plugins to get the site working as I would like.

I’m far from finished and perhaps I never will be. I feel like personal web sites change as often as people do. I’ve had some sort of online presence since the mid-90s and I’ve been tweaking and adjusting everything ever since.

As I wrote above, I have ended up sharing to each network in different ways. I do not publish here and syndicate everywhere. It isn’t all or nothing for me. It is a mixed bag. I find the nuances between the services too numerous to be able to do so. Others do a far better job. So, I reply to tweets directly on Twitter, I post things to Instagram that I do not post elsewhere (and I’m OK with that, even if it all goes away some day) and I post photos to my site that may or may not end up on Instagram or Twitter or Facebook, and I pick and choose which status updates end up being syndicated as well. It’s a bit of a mess but it is my mess.

But there is one important rule I have… anything that I want to live forever lives here.

I’m beginning to think of these networks as microphones with giant logos on them. Imagine someone giving a speech and there being a microphone for each news network in front of them. My blog is the lectern and the microphones are for Twitter and Facebook. When I want to speak into both microphones I do, when I want to speak into one I do, and when I don’t want to speak to either of them I cover them with my hand. Instagram is at a different lectern altogether because I want it to be. When I want to say something there I walk over to it.

This approach is working for me. I think one of the biggest drawbacks to only publishing on these platforms is that at any moment it can all go up in flames and you’ll have no way to recover your data or your audience (if that is important to you). By publishing the things I want to live on to my site I have control over that. For the stuff I decide to post directly to those networks I do so knowing it can (and likely will) disappear. I have peace-of-mind knowing I have a copy for myself.

I’ve mentioned before that I enjoy the fact that I’m treating Twitter so differently. That I’m sharing things that I typically wouldn’t have (example). And that I’m publishing longer posts as well.

I love my web site!

 

June 20, 2016

My #FollowFriday recommendations

Today I decided to go through the list of accounts that I follow on Twitter and cherry-pick those I think others should consider following and why. I’ve tweeted all of the suggestions but I also wanted to catalog them here on my blog.

Update, September 23, 2016:

These are just a few of the Twitter accounts that I’m currently following. The accounts that I follow change all the time. But my general goal is to have a timeline that continues to inspire me to make and share.

June 3, 2016

Twitter’s problem

I already quoted Gary this morning but I might as well do it again. In Daily Vee #29 (go right to the moment here) he said:

I know that 8 years ago if I said “follow my friend Joel, he’s awesome” 2,000 people would do that when I had 7,000 followers and now 37 will do that when I have 1.2 million followers.

This is it in a nutshell. Twitter’s “problem”. I remember in 2007 being able to tweet, at a bar in Philadelphia, “I need a beer” and someone bringing me one within a few minutes. I likely had less 200 followers then. I have 3,000+ now and no one is listening to me at all on there.

I don’t know exactly how to fix Twitter’s issues. I have ideas, of course, but I don’t know how to fix this problem; everyone is on Twitter that matters already and yet nothing is happening on there. And yet everything is. What a weird situation!

April 12, 2016

But, why did you unfollow me?

Please do not be alarmed if you’ve noticed that I have unfollowed you on the Twitter. It isn’t because I do not like you. It is, again, because I’m refactoring the way that I handle Lists on Twitter.

The unfortunate consequence of this most recent refactoring, though, is that if you have a private account I may no longer be able to follow you on Twitter. Twitter does not currently allow me to add your account to a list if we don’t follow each other.

Update: Why am I refactoring? This is why.

February 5, 2013

My island on this ocean

Me, over four years ago:

As it stands I post what I’m currently doing to Twitter, I am testing out Pownce with mobile blogging, events, links, and files, I post mobile phone photos to Flickr (as well as the occasional screenshot), videos go on Viddler, bookmarks end up on Ma.gnolia, tasting notes end up on Cork’d, and my thoughts on Appleproducts find their way toTUG.n.

What a difference four years can make! Pownce, Ma.gnolia, Cork’d, TUG.n, all gone. Flickr rarely gets my attention. Twitter is still here but is changing policies more often than I change my shirt. Viddler, I’m very proud to say, is stronger than ever but is certainly a much different service than it was then.

The Internet is like the open ocean and what we publish seems to be on a life raft simply going along for the ride.Yet our personal websites seem to be like small islands in this ocean. Sure, their beaches may change from time-to-time but the island remains – like a beacon to all travelers that we’re still here – somewhere to always come back to as these rafts take on water and eventually sink into the deep.

This environment forces me to rethink, yet again, how and where I publish on the web. This internal debate seems to be one that keeps coming up, over and over, year after year, as the ocean of the Internet ebbs and flows.

Should I simply post everything that I publish directly to this site and nowhere else? Do I cross post things to this site and also onto other services? Do I simply link back to this site from those services? Do I syndicate to those services with their own accounts (like I do now on Twitter and Facebook for this site)? Do I post some content here and some content elsewhere?

Believe it or not, and you may think I’m crazy, but these questions plague me all of the time. I constantly struggle with this. And I never seem to muster the conviction to make a hard choice and so I’ve got content everywhere; Twitter, Instagram, Facebook, the brand-new App.net, Flickr, a little on Google , and so on.

Why does it take conviction to limit myself to only posting on this site? Because there is a pull and a need to share this content with as many people as possible. With nearly 2,000 followers on Twitter, a few hundred on Instagram, friends and family on Flickr, etc. it is hard to limit the exposure of this content. I want people to see what I’m publishing. I’d be lying if I said I didn’t. This site, as it stands, only has a relatively small audience. Some of my posts get views in the single digits, others, in the hundreds of thousands. So I can never really be sure how many people are paying attention. That is why it takes conviction. I have to be OK with the fact that maybe, just maybe, no one will notice. And maybe, just maybe, no one will care.

I think I’ve gotten to that point. Even as I write this I’m coming around to the idea that I don’t really need anyone to read this post. And if they do read it I’d much rather them read it here than on Facebook or Google . Whether or not I choose to publish here on my site or elsewhere doesn’t really matter at all to anyone but me. And I want to publish to my site. So I should publish in a way that makes me happy, right?

There is an upside to making this a hard, line-in-the-sand choice. If anything I post is shared around the web it will point back to my website. My island. Some have built up enormous followings on Twitter and Instagram. What happens when they go away or change? I’d much rather people remember me for my website than for my Instagram stream.

So what does this mean? Well, I’ve thought about it. And I’m still going to tweet. Though probably far less. Twenty-five thousand plus tweets so far and counting. My entire family and most of my close friends are on Twitter. And, using Twitter Lists, I’m able to get a lot of value from this service. Far more than any other. However, I’m done with Facebook, Google , Flickr, ADN and Instagram (even though I love Instagram). Everything that I publish is going to be on this site. Follow, don’t follow, it is up to you.

Do you deal with this struggle? I’d love to read about how you’re dealing with it on Hacker News.

Some have asked if they’ll be able to stay subscribed to this site via Twitter and Facebook. Yes, you will. As long as their policies allow for it. And also RSS if you’re a nerd like me.

October 1, 2012

RSS to Twitter using PHP

Update January 19, 2010: This script is now available on GitHub. Go forth and fork.

Today I noticed that my now ancient PHP script to update Twitter automatically using PHP/cron needed to be updated. It turns out that Twitter stopped recognizing URLs with ? in them as clickable links. Here is an example tweet where you’ll notice this happening.

I could have told Twitter and asked that they update the way they handle URLs but in reality my script was old, slow, too long, and shouldn’t include ? anyway so I figured I’d write a new one from scratch that included my short URL scheme.

So, here is the PHP script to parse an RSS feed and send the posts to Twitter. It includes a caching mechanism so that you won’t have duplicate URLs posted to Twitter. If you want it, take it. However, if you are better than I am at PHP (most 6yr. olds are better than I am at programming) then I ask that you fork the script on Gist and try to improve it.

Update Dec. 6 @ 5:34p: Kyle Slattery, follow Viddler team member, loves him some Ruby on Rails. As such he’s offered up this version of the script rewritten in Ruby.

Next up we have Anthony Sterling, self-proclaimed “PHP addict”, who has rewritten the script to make the configuration a bit easier. He also changed the way the cache is saved. He’s using a hashed version of the title for each post as his key. I do not believe this to be the best way to go, since post titles can easily change after publishing – but I do like that the script is about 20 lines shorter and the code is arguably cleaner.

Thanks to both Kyle and Anthony for their versions. Lets keep this going and see if we can get this script much more succinct, stable, faster, and usable by others?

December 6, 2009