Advance The World

I mentioned a few posts back that bugs.zachgoldberg.com would come around, and after a lot of procrastination it is finally here!

First things first — a couple thank yous and relavent links are in order:

• dreamhost, for making the one-click-install so utterly painless
• edgewall for providing trac as open and free software
• The guy at 119labs for writing this blog post which detailed very nicely how to install the trac openid plugins inside a dreamhost one-click-install.  This is actually non-trivial because without dreamhost PS you do not get root or the ability to add eggs to the system, so you need to setup your own local python virtualenv to build the needed eggs.  There is also some config kung-fu that needs to be done to make the SQL schemas happy.

So, what is this bug tracker about?  We’re not concerned about software here, this is a “Bug Tracker for People.”  Did your roomate forget to buy milk when he finished the last carton?  Spouse always leaving the toilet seat up?  Not cleaning up the sink?  These are common everyday problems that we nag each other about and can actually lead to real frustration and interpersonal drama.  Finally there is an avenue to enable documented discussion about these issues and come to civil and mutually beneficial conclusions.

There are so many major corporations suing each other over complete drivel now a days that I’ve lost track of who is pointing the lawsuit-press-generator at who. Below is an aggregate from a few minutes of Googlin’.  Kudos for the idea for the title goes to the NYTimes.

(Note: this is still a work in progress)

• Motorola Sues Apple
  • When: 10/6/2010
  • Type: Patent Infringement
  • Number of Patents: 18
  • Patent Claims Cover: Signal Protocols, 802.11 (wifi), antennas, wireless email, proximity sensing, software application management, location-based services and multi-device synchronization
  • Hilarious Company Claim: “After Apple’s late entry into the telecommunications market, we engaged in lengthy negotiations, but Apple has refused to take a license. We had no choice but to file these complaints to halt Apple’s continued infringement.”
  • Responses:
  • Outcome:
  • Links:
    • http://www.engadget.com/2010/10/06/motorola-suing-apple-for-patent-infringement?icid=sphere_blogsmith_inpage_engadget

• Microsoft Sues Motorola
  • When: 10/1/2010
  • Type: Patent Infringement
  • Number of Patents: 9
  • Patent Claims Cover: Android on Motorola’s smartphones.   email & calendar synchronization, battery charging, signal strength calculation, meeting scheduling.
  • Hilarious Company Statement: “We have a responsibility to our customers, partners, and shareholders to safeguard the billions of dollars we invest each year in bringing innovative software products and services to market,”
  • Responses: None yet
  • Links:
    • http://bits.blogs.nytimes.com/2010/10/01/microsoft-activates-android-lawsuit-against-motorola/
• Oracle sues Google
• When:
• Type: Patent Infringement
• Number of Patents:
• Patent Claims Cover:
• Hilarious Company Statement:
• Responses:
• Links:
• Apple Sues HTC (and threatens Google)
  • When:  March 2010
  • Type: Patent Infringement
  • Number of Patents: 20
  • Patent Claims Cover: Interfaces, Gestures, Touchscreen Heuristics, How to Scroll On A TouchScreen, Undervolting to conserve power, “Object-Oriented Graphic System”, OS Eventing systems,
  • Hilarious Company Statement: “We can sit by and watch competitors steal our patented inventions, or we can do something about it. We’ve decided to do something about it,” said Steve Jobs, Apple’s CEO. “We think competition is healthy, but competitors should create their own original technology, not steal ours”
  • Responses:
    • Google: “We are not a party to this lawsuit. However, we stand behind our Android operating system and the partners who have helped us to develop it”
  • Outcome: None yet
  • Links:
    • http://www.engadget.com/2010/03/02/apple-vs-htc-a-patent-breakdown/
    • http://www.osnews.com/story/22951/Google_HTC_Respond_to_Apple_Lawsuit
• Apple Sues Nokia
  • When:  9/28/10
  • Type: Patent Infringement
  • Number of Patents: 13
  • Hilarious Company Statement: “”Other companies must compete with us by inventing their own technologies, not just by stealing ours,” Bruce Sewell, Apple’s general counsel, said at the time.”
  • Links:
    • http://www.bbc.co.uk/news/technology-11435639
• Nokia Sues Apple
  • When: October 2009
  • Type: Patent Infringement
  • Number of Patents: 10
  • Patent Claims Cover: wireless data, speech coding, security and encryption.
  • Hilarious Company Statement: “Apple’s refusal to respect Nokia’s intellectual property and attempt to free ride on the back of Nokia’s innovation,” Nokia said in a statement”
  • Links:
    • http://www.bbc.co.uk/news/technology-11435639
    • http://www.mobileburn.com/news.jsp?Id=10990
• Apple is sued for Cover Flow by Mirror Worlds
  • When: 2008
  • Outcome: Apple owes $208.5 Million
  • Links:
    • http://www.tuaw.com/2010/10/04/ouch-apple-dinged-for-208-5-million-in-patent-infringement-cas?icid=sphere_blogsmith_inpage_engadget
    • http://www.nytimes.com/2010/10/05/technology/05apple.html?_r=2

That is, unless you plan on owning them and supporting them.

Allow me to explain. There are some tools, like the Linux Kernel or the Python Interpreter that you’ll be able to use (for the most part…) without a second thought. (Though, there are definitely imaginable, and in fact scenarios I’ve run into, where tweaking and investigation of these tools is required).

No, for the most part I want you to be wary of niche tools. Things like the Django framework, the Zenoss monitoring suite, random python libraries, system maintenance tools etc. Don’t get me wrong, these are fantastic pieces of software which are each excellent in their own right. But if you chose to use them in your business or in a project consider whether or not you’re willing to maintain the tool as well; could you get away with something inhouse that is much smaller, with many fewer features, but is tailored to exactly your needs?

Why? Well these tools each have use cases in mind. I bet dollars to donuts your specific use case will align with the use case of the tool, but not quiet perfectly. You’ll eventually do somethng ever so slightly out of scope. The tool will seem like it’ll work fine, but soon you’ll find that it is not perfect and you’ll have to hack it in some way, change a few lines of code here and there, add some extra script to support an ancillary function, run a cron job to periodically clean logs etc.

And that is fine. It is expected that you’ll need to do some tool sharpening.

But you probably won’t realize how much time you’ll sink into these tools. Some examples in my recent history: The Zenoss Monitoring Suite. A fantastic tool for monitoring cloud infrastructure. But what happens when you’re scaling your busienss at 200% or more per quarter? Zenoss worked fine with 20 machines, but how does it handle 200, or 2000? Ordinarily sure Zenoss can do 2000 machines, but we started adding custom templates to the nodes to collect extra data tailored to our application. Pretty quickly we found Zenoss either running out of system resources (we’ve moved it to larger boxes several times), or falling behind entirely or flat our having bugs which cause a certain use case to not work (Maintenance windows don’t quiet work as well as one would hope…). We’ve spent all in all hundreds of developer hours just working out ways to make Zenoss work for us. At the end of the day we’re finding that many of our original use cases can actually be handled perfectly fine with independent monitoring scripts entirely separate and outside of Zenoss; avoiding a lot of headache overhead. I can come up with at least a dozen other examples for other tools I’ve used including but not limited to the Linux Kernel itself, the python interpreter (who hasn’t had GIL or memory problems?), Django, Django-Piston, the Hudson Continuous Integration Environment, the Graphite graphing/monitoring tool, puppet, iclassify, nfs, nis, etc.

So, support your tools. Contribute your changes back upstream to help out the next person who happens to run into your use case. Just don’t try and support more than you can.

I’ll end this rant on one final point. Obviously you can only really make use of tools you can support, and you can only support tools you have the source and capability to build. Another great reason to never use proprietary software!

Today Google had one of its fancy playable logos. They really are nifty – I actually found myself playing with it and admiring the physics for a couple of minutes. And then I noticed…. So was everybody else. I got a couple of messages from friends who also spent a couple of minutes with it. That got me thinking… How much productivity does the world lose when Google does this?

To make life simpler, let’s just think about the USA.

According to Internet World Stats there are 230 Million internet users in the USA alone. Lets be conservative and say 10% of them saw the google.com logo today and spent an average of 2 minutes playing with it. (though, given that every user probably makes at least one search during their online session and Google has a 70%+ market share in search thats probably very very conservative).

23 Million people each spending 2 minutes = 46 Million Minutes = 767,000 Man-Hours = 31900 Days = 87 man-years of human productivity.

According to Wired.com all of Wikipedia took roughly 100 Million man hours to create. So, Google’s fancy logo consumed about as much time as creating 1% of wikipedia, in the United States. Given that the US actually only accounts for a moderate amount of total internet traffic (and assuming Google used the fancy logo in every country) that number could really be 5 or 10%.

The Empire State building was built in 7 Million man hours, so in the US alone we could’ve put up 10.2 floors (its 102 stories) of the Empire State building!

Now, remember the pacman logo? I definitely spent more than 2 minutes on that… combine both logos and you have a whole Empire State Building

The biggest challenge in helping out a friend in need (running Linux, of course — I don’t bother troubleshooting any Windows problems with any tool other than an Ubuntu Live CD now-a-days) is often their local networking setup. Why? Because I can point out tutorials or guides until I am blue in the face but some people are just very uncomfortable reconfiguring their system and insist they will ‘mess it up’, so sometimes you have to just do it help fix things yourself and firewalls get in the way.

The quick thing to do is SSH into their machine, or VNC in, and just troubleshoot the problem yourself. However, 9 times out of 10 a router, or a NAT, or a firewall, or a proxy, or an act of god will prevent you from establishing a TCP connecton to the machine. To get around this problem most commercial troubleshooting applications require the user to initiate the connection. So, I figure, why can’t we use some quick open source tunnel magic to do the same?

Here is my quick and handy guide to “hacking in” (aka (reverse) ssh port forwarding) to somebody’s PC that is behind a firewall or NAT:

0) You need yourself a machine which has an open ssh port (22 by default) to the internet for this to work (herein referred to as myopenpc.com). It doesn’t have to be your machine, it can be a third party machine that you have a login to. All it needs is to be running openssh on it. A Linode.com node, or a dreamhost shell account, anything like these will do. I personally use a cluster I still have access to at my undergraduate college the University of Penn (yay Force@STWing!).

You’ll also need a login on <YOUROPENPC>.com (herein: myopenpc.com) for your friend (or they can borrow yours) as well as a login on your friend’s machine (duh) and preferrably also root on your friend’s machine (if only temporary).

1) When you first help your friend setup their computer add an extra desktop shortcut for them which will run the following command (or walk them through typing in this command)

ssh -C -R 8888:localhost:22 FRIENDSUSERNAME@<YOUROPENPC>.com

When your friend runs this command it will prompt for a password. Now you can either give them their own username and password on the open-to-the-world machine or you can temporarily change your password (if you dont mind giving them access to your account… assuming they have no idea what a terminal is I find this option works fine for most cases).

So, what does this do? “-R” means “Reverse Port Foward” — When SSH makes the connection to the “myopenpc.com” server it sets up an extra port on “myopenpc.com” (port 8888) to automagically redirect back to port 22 on localhost. Therefore any data sent to port 8888 on myopenpc.com actually is rerouted over the original ssh connection to port 22 on localhost. (localhost in this case means your friend’s computer).

(-C just means enable compression. If you’re going to do VNC forwarding (see below) this will help keep bandwidth usage down and speed up)

2) Next, you need to login to myopenpc.com:

ssh me@myopenpc.com

3) From here, you can now do:

ssh -p 8888 username_on_friends_computer@localhost

and Voila! You’re now logged into your friend’s computer despite any firewall or NAT they might have!

Alright, but what if you REALLY need access to their X Session (aka what they see on their screen) to help with the issue? We can fix that too.

4) Install x11vnc and run it on your friend’s PC (which you can now do because you’re logged into their machine)

sudo apt-get install x11vnc && x11vnc -display :0

5) x11vnc will by default host a vnc session on port 5900. We, however, only have access to port 22 on our friend’s computer (remeber the :22 in the initial ssh -R command?). Nothing is stoping us from running another ssh session with a new port number though! So, from your friend’s PC, log into myopenpc.com again!

ssh -C -R 8889:localhost:5900 FRIENDSUSERNAME@myopenpc.com

Now we have two different ssh connections from your friend’s computer to myopenpc.com. One sets up port 8888 to redirect back to your friend’s computer on port 22 (for ssh connections) and one redirects back to your friends computer on port 5900 (for VNC). You could change this to any port you want — 80 for HTTP if your friend has a webserver, 631 if you want to access their CUPS printing page etc.

6) We’re not quiet done, however. Only myopenpc.com port 8889 can talk to the x11vnc server on your friend’s machine. Unless you have an X session with a vncviewer on myopenpc.com you still can’t connect. So, what we now need to do, is connect a port on YOUR machine to port 8889 on myopenpc.com. This is a “forward” port forward and can be done quiet simply.

ssh -C -L 9999:localhost:8889 me@myopenpc.com

Now any data sent to port 9999 on your machine will actually be routed via ssh to port 8889 on myopenpc.com. Any data sent to port 8889 on myopenpc.com actually is routed via another ssh session to port 5900 on your friend’s computer. Ah, what a tangled web we weave!

7) We’re almost done! Now, all we need to do is setup the vnc session by pointing to localhost port 9999 (which redirects to myopenpc.com which redirects to your friend’s computer)

vncviewer localhost:9999

Voila!

Recap of what we covered:

• ssh -R — Reverse port forwarding. This allows you to setup a port on the REMOTE machine which points to your LOCAL machine.
• ssh -L — Normal port forwarding. This allows you to setup a port on the LOCAL machine which points to the REMOTE machine.
• ssh -C — Enable compression in your ssh connections.
• Port 22 — what ssh uses by defualt
• Port 5900 — what VNC uses by default
• Port 631 — what CUPS (Linux/Unix printing services) uses by default
• x11vnc — a program which starts a VNC session based on an already running X session
• vncviewer — a standard program to connect to a VNC session.

So I admit it,  I have a huge problem. I check my email constantly — every 20 seconds constantly. I cannot work on a computer with less than two screens because email must always be visible.  Buzz and other social media, however, don’t get checked as often.  So, today,  right before a late lunch I finally get around to checking Buzz — theres one message.

It was a “retweet” — Richard Stallman (THE Richard Stallman) was coming to Google for lunch today. I immediately checked the clock on my PC — 1 PM. Crap! I might be too late! I immediately forgot what I was working on, jumped out of my chair, mentioned it to a bunch of coworkers (while walking towards the door) and ran to the cafeteria.

I got there and found him towards the back (where they said they would be) — the only surprising thing was how small of table they had! There were but two other people talking to RMS, one of whom was the guy who coordinated his visit. I would think that of all places Google has a whole bunch of open source / GNU folks who would kill for a chance to talk to RMS. I understand if people might have missed the Buzz but he is kind of a memorable face, what with the big beard and all. If I was in the middle of my sandwich and looked up and saw RMS, or Linus Torvalds or Barack Obama or Larry Lessig or <insert other figure at the head of some political/ethical/technical movement you care about> at the next table with 4 empty chairs you can bet that I won’t be finishing that sandwich in the chair I started it in.

So… where was everybody?  Why did the folks I mentioned this to on the way up not come (they knew who he was)?  Why would all the engineers in the cafeteria not stop by?  To these questions I have no answers.

Talking with RMS obviously got me more motivated than I already was to promote software and technical freedom, so I grabbed some “iBad” stickets (promoting awareness of the rights Apple doesn’t give you and the evil things they do) and brought them back to the office.  I talked with some of the guys… and I know im not RMS and I don’t have 15 years experience trying to convince people of this stuff, but I’ll be blatent: I struck out entirely.  Didn’t convince anybody of anything other than the fact that I’m a nut.

So, why is this so hard?  The first problem is I am probably terrible at it and don’t have much training or experience in preaching people on ethics.  The second problem is  that the people I was talking to already dedicated Apple users, if not even “fanboys”.   I would take it to be a very difficult problem to convince some body to give up devices they already own and use — and might even do a better job than the competition — for intangible ethical reasons.

I guess I’ll just have to keep trying.

I never thought it would happen.

My girlfriend figured something out about the Internet.

In the real world I have to pretend to be normal and communicate with people.  I usually can take solice in the internet, those hallowed sacred grounds only a real hacker understands.  I can go there and know that unless I venture to the kingdom of far far away land (facebook, myspace etc.) I can be safe from the heathens.

As of today the hord has made their move on alliance territory.  The Penguin home base is no longer safe from the dreaded… girlfriend.

It all started one night when I got home very late from work (where I get to play with Linux all day… who would ever go home?).  I got the usual “you need to pay more attention to me” and “Linux will never have sex with you!”.  I sat through it all and when it was over she went to sleep and I…. opened up my Laptop (running Ubuntu Linux, of course) and started hacking.  All is right with the world.

So this story and ones like it have been happening on and off for a while.  This morning I show up at work and somebody sends me a link to the gnome bugzilla.  I’m thinking to myself… the guys at work arn’t Gnome hackers… what could this….. oh…. sh**.

I then find out the bug made it to LWN Quote of the Week (next to Richard Stallman) and Hacker News which made it to geek.com which made it to slashdot.  I never imagined my entry into the “slashdot number of zero” (think Erdos or Kevin Bacon) club would happen in this way.

This adventure is not without its benefits, however.  Many of the comments on the bug recommend that she actually try and take part in the hobby — see the light and willingly embrace the Linux love.  Maybe that’s the solution to all my troubles!

Here is to the newfound communication in our relationship via bugtrackers.  I’m going to open a personal bugtracker — bugs.zachgoldberg.com where we will continue to file bug reports on one another.  If you want your own component in the bugtracker to help you and your loved one vent your problems all you need to do is ask!

So yesterday I got a desperate call from the folks — Windows got gunked with adware, viruses and other forms of badness and dad couldn’t use it to work from home anymore. So, after work i got on a train and headed back to the ‘rents for some quality computer troubleshooting family time.

My instinct is usually to throw such a computer out the window and start over from scratch with a more…. Free… Operating system. I gave a go at cleaning it anyway and spent 30 minutes playing the usual hide and seek game with the malware. I got rid of most of it, including all the annoying popups, but things like control panel applets still wouldn’t open. I also got the worst headache of my life and wanted to shoot myself in the foot.

So I decided to go with what my gut told me from the getgo. Whipped out a usb key, backed up the important data, pulled out he livecd and got ready to start from scratch.

Lets see just what we’re getting into:

• Mom has an iPhone. She likes to buy music on iTunes and must have her contacts, music and photos synced with the desktop.
• Mom also needs Google Chrome as thats what she has used to get her aol mail for years.
• Dad has a blackberry. He just wants everything, including contacts, backed up and his phone to charge over usb.
• Both mom and dad need to be able to print and scan from the big HP multiwhatever thing that’s on their desk.

Install worked wonderfully. It’s quiet satisfying to tell the partitioning tool to “use the whole disk and wipe out any old operating systems”. 15 minutes later we’re in the desktop and I start the install, configure, google for help cycle.

Chrome is easy, a one click install and IO can drag the icon to the desktop for high”findability”.

Next, printing. This was completely automatic and ‘just worked’ out of the box. Awesome.

Next, iPhone. I hate these things. Anyway, first thought was to plug it in then try rhythm box. No dice. A quick google turns up that the device first needs to be unlocked before i can plug it in. Easy enough. Plugged it in again and now RB shows the phone, nice. I select all the files on the phone and drag them to the ‘music’ folder, it starts copying. Cool. I do the same thing to see what happens, and i get an endless cycle of popup dialogs asking me about overwriting files. Not good. Bug filed for poor User Experience.

F-spot also automatically showed up and offered to backup all the photos. Worked perfectly.

Ok, mom taken care of, blackberry time. Some quick googling found me this barrysync thing, which seemed simple enough. A quick apt-get later and ive got a really dinky looking app sitting on my screen which says no devices found, bummer. I did a bunch of googling, no dice. In desperation i went for the windows option — a reboot. Upon return to the land of the living the app actually found the device! Click ‘backup’ and 15 seconds later everything is backed up. I think.

All in all Ubuntu was able to meet the needs of my parents on day one with virtually no pain or terminal work whatsoever.  We’ll see how long they stay happy.

I’m on the subway in ny , heading home from GUADEC in The Hague, n
Netherlands. First things first — a huge thank you to the gnome foundation for helping sponsor my attendance, and to all the conference’s fantastic sponsors.

Some highlights:
-Produced a introspection->docbook documentation generator which is in a branch on gobject introspection, potentially replacing the gtkdoc c scanner.
-Keyword argument support in pygobject is en route.
-Gnome Shell looks awesome
-People seemed interested in general in introspection, which is fantastic.
-’Super Platforms’ seemed to steal the show. Technologies like DBus and Telepathy are, rightfully, the talk of the town and people are doing awesome things with them.
-Some hackers are weird, others are some of the smartest people I’ve ever met.
-LOTS of N900s… Although people in general were not as excited about N900/Maemo/Meego as I would’ve expected. Somewhat contradictory there, the ubiquity of both the device and at the same time the dislike of the device. Or perhaps the two just go hand in hand.

Some lowlights:

-I really did not enjoy the ‘anti-canonical’ vibe I got throughout the conference. I felt people were more upset at Canonical than KDE or Microsoft, which is weird for a Gnome summit. Less negativity across the board would be nice.

-Europe is far away, perhaps I’ll find a way to make it to the Boston summit, a bit closer!

-Everybody seems to be falling in love with the web, which is nice, but I don’t believe that javascript will come to dominate the desktop — simply because devs like to chose their language (and many will not chose javascript if given the choice, myself included).

All in all GUADEC was a ton of fun and very educational for me. It’s also nice to network with some of the really famous people in this space who have done things like write pulseaudio (<3 Lenhart) or the Shell (Colin’s mom).

Cheers!

I have been debugging lately with Matt Price over on Launchpad an issue with a machine using the r8192se driver would periodically crash with a kernel panic.  We have narrowed it down to an ioctl which is most commonly run by the command “iwpriv”.  A quick fix hack is to just entirely neuter the iwpriv command.

1) sudo mv /sbin/iwpriv /sbin/iwpriv.old
2) sudo echo ‘#!/bin/bash’ > /sbin/iwpriv
3) sudo chmod +x /sbin/iwpriv

You can read more about the fix and our work to uncover the bug on the bug report.