How to VNC into any Linux machine (or how I ‘hack into’ my friends computers)

The biggest challenge in helping out a friend in need (running Linux, of course — I don’t bother troubleshooting any Windows problems with any tool other than an Ubuntu Live CD now-a-days) is often their local networking setup. Why? Because I can point out tutorials or guides until I am blue in the face but some people are just very uncomfortable reconfiguring their system and insist they will ‘mess it up’, so sometimes you have to just do it help fix things yourself and firewalls get in the way.

The quick thing to do is SSH into their machine, or VNC in, and just troubleshoot the problem yourself. However, 9 times out of 10 a router, or a NAT, or a firewall, or a proxy, or an act of god will prevent you from establishing a TCP connecton to the machine. To get around this problem most commercial troubleshooting applications require the user to initiate the connection. So, I figure, why can’t we use some quick open source tunnel magic to do the same?

Here is my quick and handy guide to “hacking in” (aka (reverse) ssh port forwarding) to somebody’s PC that is behind a firewall or NAT:

0) You need yourself a machine which has an open ssh port (22 by default) to the internet for this to work (herein referred to as myopenpc.com). It doesn’t have to be your machine, it can be a third party machine that you have a login to. All it needs is to be running openssh on it. A Linode.com node, or a dreamhost shell account, anything like these will do. I personally use a cluster I still have access to at my undergraduate college the University of Penn (yay Force@STWing!).

You’ll also need a login on <YOUROPENPC>.com (herein: myopenpc.com) for your friend (or they can borrow yours) as well as a login on your friend’s machine (duh) and preferrably also root on your friend’s machine (if only temporary).

1) When you first help your friend setup their computer add an extra desktop shortcut for them which will run the following command (or walk them through typing in this command)

ssh -C -R 8888:localhost:22 FRIENDSUSERNAME@<YOUROPENPC>.com

When your friend runs this command it will prompt for a password. Now you can either give them their own username and password on the open-to-the-world machine or you can temporarily change your password (if you dont mind giving them access to your account… assuming they have no idea what a terminal is I find this option works fine for most cases).

So, what does this do? “-R” means “Reverse Port Foward” — When SSH makes the connection to the “myopenpc.com” server it sets up an extra port on “myopenpc.com” (port 8888) to automagically redirect back to port 22 on localhost. Therefore any data sent to port 8888 on myopenpc.com actually is rerouted over the original ssh connection to port 22 on localhost. (localhost in this case means your friend’s computer).

(-C just means enable compression. If you’re going to do VNC forwarding (see below) this will help keep bandwidth usage down and speed up)

2) Next, you need to login to myopenpc.com:

ssh me@myopenpc.com

3) From here, you can now do:

ssh -p 8888 username_on_friends_computer@localhost

and Voila! You’re now logged into your friend’s computer despite any firewall or NAT they might have!

Alright, but what if you REALLY need access to their X Session (aka what they see on their screen) to help with the issue? We can fix that too.

4) Install x11vnc and run it on your friend’s PC (which you can now do because you’re logged into their machine)

sudo apt-get install x11vnc && x11vnc -display :0

5) x11vnc will by default host a vnc session on port 5900. We, however, only have access to port 22 on our friend’s computer (remeber the :22 in the initial ssh -R command?). Nothing is stoping us from running another ssh session with a new port number though! So, from your friend’s PC, log into myopenpc.com again!

ssh -C -R 8889:localhost:5900 FRIENDSUSERNAME@myopenpc.com

Now we have two different ssh connections from your friend’s computer to myopenpc.com. One sets up port 8888 to redirect back to your friend’s computer on port 22 (for ssh connections) and one redirects back to your friends computer on port 5900 (for VNC). You could change this to any port you want — 80 for HTTP if your friend has a webserver, 631 if you want to access their CUPS printing page etc.

6) We’re not quiet done, however. Only myopenpc.com port 8889 can talk to the x11vnc server on your friend’s machine. Unless you have an X session with a vncviewer on myopenpc.com you still can’t connect. So, what we now need to do, is connect a port on YOUR machine to port 8889 on myopenpc.com. This is a “forward” port forward and can be done quiet simply.

ssh -C -L 9999:localhost:8889 me@myopenpc.com

Now any data sent to port 9999 on your machine will actually be routed via ssh to port 8889 on myopenpc.com. Any data sent to port 8889 on myopenpc.com actually is routed via another ssh session to port 5900 on your friend’s computer. Ah, what a tangled web we weave!

7) We’re almost done! Now, all we need to do is setup the vnc session by pointing to localhost port 9999 (which redirects to myopenpc.com which redirects to your friend’s computer)

vncviewer localhost:9999

Voila!

Recap of what we covered:

  • ssh -R — Reverse port forwarding. This allows you to setup a port on the REMOTE machine which points to your LOCAL machine.
  • ssh -L — Normal port forwarding. This allows you to setup a port on the LOCAL machine which points to the REMOTE machine.
  • ssh -C — Enable compression in your ssh connections.
  • Port 22 — what ssh uses by defualt
  • Port 5900 — what VNC uses by default
  • Port 631 — what CUPS (Linux/Unix printing services) uses by default
  • x11vnc — a program which starts a VNC session based on an already running X session
  • vncviewer — a standard program to connect to a VNC session.

Curious Behavior

So I admit it,  I have a huge problem. I check my email constantly — every 20 seconds constantly. I cannot work on a computer with less than two screens because email must always be visible.  Buzz and other social media, however, don’t get checked as often.  So, today,  right before a late lunch I finally get around to checking Buzz — theres one message.

It was a “retweet” — Richard Stallman (THE Richard Stallman) was coming to Google for lunch today. I immediately checked the clock on my PC — 1 PM. Crap! I might be too late! I immediately forgot what I was working on, jumped out of my chair, mentioned it to a bunch of coworkers (while walking towards the door) and ran to the cafeteria.

I got there and found him towards the back (where they said they would be) — the only surprising thing was how small of table they had! There were but two other people talking to RMS, one of whom was the guy who coordinated his visit. I would think that of all places Google has a whole bunch of open source / GNU folks who would kill for a chance to talk to RMS. I understand if people might have missed the Buzz but he is kind of a memorable face, what with the big beard and all. If I was in the middle of my sandwich and looked up and saw RMS, or Linus Torvalds or Barack Obama or Larry Lessig or <insert other figure at the head of some political/ethical/technical movement you care about> at the next table with 4 empty chairs you can bet that I won’t be finishing that sandwich in the chair I started it in.

So… where was everybody?  Why did the folks I mentioned this to on the way up not come (they knew who he was)?  Why would all the engineers in the cafeteria not stop by?  To these questions I have no answers.

Talking with RMS obviously got me more motivated than I already was to promote software and technical freedom, so I grabbed some “iBad” stickets (promoting awareness of the rights Apple doesn’t give you and the evil things they do) and brought them back to the office.  I talked with some of the guys… and I know im not RMS and I don’t have 15 years experience trying to convince people of this stuff, but I’ll be blatent: I struck out entirely.  Didn’t convince anybody of anything other than the fact that I’m a nut.

So, why is this so hard?  The first problem is I am probably terrible at it and don’t have much training or experience in preaching people on ethics.  The second problem is  that the people I was talking to already dedicated Apple users, if not even “fanboys”.   I would take it to be a very difficult problem to convince some body to give up devices they already own and use — and might even do a better job than the competition — for intangible ethical reasons.

I guess I’ll just have to keep trying.

Zach & RMS

HELP! My Girlfriend Learned How To Use A Bug Tracker


I never thought it would happen.

My girlfriend figured something out about the Internet.

In the real world I have to pretend to be normal and communicate with people.  I usually can take solice in the internet, those hallowed sacred grounds only a real hacker understands.  I can go there and know that unless I venture to the kingdom of far far away land (facebook, myspace etc.) I can be safe from the heathens.

As of today the hord has made their move on alliance territory.  The Penguin home base is no longer safe from the dreaded… girlfriend.

It all started one night when I got home very late from work (where I get to play with Linux all day… who would ever go home?).  I got the usual “you need to pay more attention to me” and “Linux will never have sex with you!”.  I sat through it all and when it was over she went to sleep and I…. opened up my Laptop (running Ubuntu Linux, of course) and started hacking.  All is right with the world.

So this story and ones like it have been happening on and off for a while.  This morning I show up at work and somebody sends me a link to the gnome bugzilla.  I’m thinking to myself… the guys at work arn’t Gnome hackers… what could this….. oh…. sh**.

I then find out the bug made it to LWN Quote of the Week (next to Richard Stallman) and Hacker News which made it to geek.com which made it to slashdot.  I never imagined my entry into the “slashdot number of zero” (think Erdos or Kevin Bacon) club would happen in this way.

This adventure is not without its benefits, however.  Many of the comments on the bug recommend that she actually try and take part in the hobby — see the light and willingly embrace the Linux love.  Maybe that’s the solution to all my troubles!

Here is to the newfound communication in our relationship via bugtrackers.  I’m going to open a personal bugtracker — bugs.zachgoldberg.com where we will continue to file bug reports on one another.  If you want your own component in the bugtracker to help you and your loved one vent your problems all you need to do is ask!

A true test of Ubuntu: Setting it up for the parents in 2 hours

So yesterday I got a desperate call from the folks — Windows got gunked with adware, viruses and other forms of badness and dad couldn’t use it to work from home anymore. So, after work i got on a train and headed back to the ‘rents for some quality computer troubleshooting family time.

My instinct is usually to throw such a computer out the window and start over from scratch with a more…. Free… Operating system. I gave a go at cleaning it anyway and spent 30 minutes playing the usual hide and seek game with the malware. I got rid of most of it, including all the annoying popups, but things like control panel applets still wouldn’t open. I also got the worst headache of my life and wanted to shoot myself in the foot.

So I decided to go with what my gut told me from the getgo. Whipped out a usb key, backed up the important data, pulled out he livecd and got ready to start from scratch.

Lets see just what we’re getting into:

  • Mom has an iPhone. She likes to buy music on iTunes and must have her contacts, music and photos synced with the desktop.
  • Mom also needs Google Chrome as thats what she has used to get her aol mail for years.
  • Dad has a blackberry. He just wants everything, including contacts, backed up and his phone to charge over usb.
  • Both mom and dad need to be able to print and scan from the big HP multiwhatever thing that’s on their desk.

Install worked wonderfully. It’s quiet satisfying to tell the partitioning tool to “use the whole disk and wipe out any old operating systems”. 15 minutes later we’re in the desktop and I start the install, configure, google for help cycle.

Chrome is easy, a one click install and IO can drag the icon to the desktop for high”findability”.

Next, printing. This was completely automatic and ‘just worked’ out of the box. Awesome.

Next, iPhone. I hate these things. Anyway, first thought was to plug it in then try rhythm box. No dice. A quick google turns up that the device first needs to be unlocked before i can plug it in. Easy enough. Plugged it in again and now RB shows the phone, nice. I select all the files on the phone and drag them to the ‘music’ folder, it starts copying. Cool. I do the same thing to see what happens, and i get an endless cycle of popup dialogs asking me about overwriting files. Not good. Bug filed for poor User Experience.

F-spot also automatically showed up and offered to backup all the photos. Worked perfectly.

Ok, mom taken care of, blackberry time. Some quick googling found me this barrysync thing, which seemed simple enough. A quick apt-get later and ive got a really dinky looking app sitting on my screen which says no devices found, bummer. I did a bunch of googling, no dice. In desperation i went for the windows option — a reboot. Upon return to the land of the living the app actually found the device! Click ‘backup’ and 15 seconds later everything is backed up. I think.

All in all Ubuntu was able to meet the needs of my parents on day one with virtually no pain or terminal work whatsoever.  We’ll see how long they stay happy.

GUADEC 2010 debrief

I’m on the subway in ny , heading home from GUADEC in The Hague, n
Netherlands. First things first — a huge thank you to the gnome foundation for helping sponsor my attendance, and to all the conference’s fantastic sponsors.

Some highlights:
-Produced a introspection->docbook documentation generator which is in a branch on gobject introspection, potentially replacing the gtkdoc c scanner.
-Keyword argument support in pygobject is en route.
-Gnome Shell looks awesome
-People seemed interested in general in introspection, which is fantastic.
-‘Super Platforms’ seemed to steal the show. Technologies like DBus and Telepathy are, rightfully, the talk of the town and people are doing awesome things with them.
-Some hackers are weird, others are some of the smartest people I’ve ever met.
-LOTS of N900s… Although people in general were not as excited about N900/Maemo/Meego as I would’ve expected. Somewhat contradictory there, the ubiquity of both the device and at the same time the dislike of the device. Or perhaps the two just go hand in hand.

Some lowlights:

-I really did not enjoy the ‘anti-canonical’ vibe I got throughout the conference. I felt people were more upset at Canonical than KDE or Microsoft, which is weird for a Gnome summit. Less negativity across the board would be nice.

-Europe is far away, perhaps I’ll find a way to make it to the Boston summit, a bit closer!

-Everybody seems to be falling in love with the web, which is nice, but I don’t believe that javascript will come to dominate the desktop — simply because devs like to chose their language (and many will not chose javascript if given the choice, myself included).

All in all GUADEC was a ton of fun and very educational for me. It’s also nice to network with some of the really famous people in this space who have done things like write pulseaudio (<3 Lenhart) or the Shell (Colin's mom).

Cheers!

Sync Multiple Calendars on the Nokia N900

I recently moved to a different corporate infrastructure (you can guess from where to where by reading a bit about the company I work for, Invite Media :)) and decided that I was going to be serious about using the calendar on my Nokia N900.  I had the following “desires”:

The Problem:

  • Work and Pleasure should be separate.    I want my work stuff on my corporate google calendar and I want my personal apts and reminders on my personal google calendar.  No mixing.
  • I want both to sync to the phone at a reasonable interval and not screw up either calendar or cross events from one to the other etc.
  • I would like to not pay anything to accomplish this.

I noticed something really quickly:

Stock Maemo 5 cannot support syncing calendar etc. with more than one Mail For Exchange account.

That sucks.  Oh well, the Maemo-ians must’ve run out of time and had to cut scope on the built in MFE syncing.  Worse things have happened in the world.

My Solution (Update: This may not be the best way to do it.  As is pointed out in the comments another application, Erming, exists which purports to be able to solve this problem.    I’m going to test it soon.)

  • TWO goosync accounts.  I ended up paying for one and using the free service for the second.  I may sadly have to end up paying for both.  (This really stinks :().  I used goosync because it does a very good job of exposing google calendar as a SyncML service.  I couldn’t find anything comparable to this on the web.  The reason you need two accounts is that, even though Goosync supports syncing multiple calendars, all the calendars have to come from the same google login.  I have two in this case.  Although, now that I am typing this, I realize I could just share one calendar with the other account and sync them that way.  That would mean that both calendars show up as the same color and mix on the device though, which would be suboptimal.  If thats OK with you that might be one route.
  • Syncevolution (in extras-testing) on the device to sync with the two different goosync accounts.  I setup each goosync account to use a different Maemo 5 calendar so they appear differently and I can create new events and chose which calendar they go to (very nice!)
  • Syncevolution by default does not provide more than a once daily sync.  Thankfully the Maemo 5 incarnation of syncevolution is really just a UI ontop of an underlying command line client, so we can use the command line client to trigger a sync whenever we want.  I used alarmd (available in extras-devel) to do this.  I used a tip from here: http://www.estamos.de/blog/2009/05/08/running-syncevolution-as-cron-job/ to setup the alarm command.  To save you the jump its:
    • env `dbus-launch` sh -c ‘trap “kill $DBUS_SESSION_BUS_PID” EXIT; syncevolution <NAME_OF_SYNCEVOLUTION_ACCOUNT>’
    • I setup two alarms, one for each goosync account to run every hour and it has been working flawlessly for more than a week now!  If you pay for goosync you can probably use this for tasks and notes as well (I havn’t tested this yet)

UPDATE: 
  1. I had no idea that I was syndicated on planet Maemo for my #Maemo posts.  Cool.
  2. Apparently there is another application which I had overlooked, Erming-NG.  It purports to be able to sync directly with multiple Google Calendars.  I will investigate this further!  
  3. I swapped N900s and had trouble resetting up Goosync.  I tried out erming and it worked flawlessly without any middle man in connecting to my various Google calendars — the only problem is that I couldn’t get it to sync more than once per day (and the connection needs to already be active for the sync, it wont auto-connect to wifi or 3g for its sync.  It actually throws obnoxious errors if its not connected).

Back from France

Salut mes amis!

I’m rather tempted to write this in French; given that for the past two weeks I have been travelling around “Ce Beau Pays”.

Since you’re not really interested in what I did in France (touring with my girlfriend mostly) I’ll instead update on some more relevant things:

  • The startup company I’ve been devoting my life to these past 18 months, invitemedia, was just acquired by a small company in California you might have heard of: Google.
  • I had previously had an offer to work at Google as an Associate Product Manager (APM).
  • In short: I work now for Google as part of Invite Media, and will continue to work for Google in the future. Where and under which program is TBD.
  • I will be attending Guadec 2010 in The Hague, Netherlands!
  • I am submitting two proposals for lightning talks at Guadec.
  • Zhaan version 0.5.1 is now in Extras-Testing, waiting for your QA-ing pleasure!
  • I am moving to NYC and still looking for apartments on the west side.

I am going to spend this weekend unpacking and continuing to figure out short term things in life, like where I will be living come July.  Zhaan work, Pygi work and lightning talk proposals will.

PyGi version 0.5, the ‘Watch out, theres a Volcano!’ release

(Special note for this version of the release statement on this blog — All the features Zhaan needed made it into the release, hence Zhaan is 100% compatible with PyGi 0.5)

It is my pleasure to bring to the world the first release of PyGi –
the Python GObject Introspection bindings – version 0.5. We chose the
number 0.5 to indicate that we believe PyGi to be of beta quality. We
want to encourage the brave souls of the world to try it out and help
us by reporting any bugs that are found.

PyGi is nearly feature complete. A highly condensed list of features:

– Implements in and out arguments of all types
– Implements closures, callbacks and virtual functions
– Implements wrapping of structs, objects and interfaces.

A huge thank you is due to several people for making this happen.

– Simon van der Linden, for all of his hard work on PyBank/PyGi.
– Tomeu Vizoso for all of his hard work on PyGi throughout the past
months/years
– John (J5) Palmieri, Colin Walters, Johan Dahlin and others for
assistance, contributions and help along the way,
– All of the great folks who participated in the Gnome/Python 2010
hackfest who made this release possible (including generous support
from Red Hat, Canonical and OLPC)

Version 0.6 of PyGI is likely to include:
– Python 3.1 support (already 95% working! Thanks David Malcom and
John Ehresman!)
– A major refactoring of some of the thickest parts of the code
– Much improved documentation
– Improved compatibility with older bindings

PyGi 0.5 is available for download here:

http://ftp.gnome.org/Public/gnome/sources/pygi/0.5/pygi-0.5.tar.gz

You can find the PyGi website/wiki:

http://live.gnome.org/PyGI

And the public git repo at:

git://git.gnome.org/pygi

Bug tracking is done in project pygi on bugzilla.gnome.org.

Cheers,
-Zach Goldberg

PyGi Maintainers:
Simon van der Linden
Tomeu Vizoso
Zach Goldberg

Git Shortlog:

Alex Dedul (1):
Search for python-config-${VERSION} when python${VERSION}-config
is not found

Anderson Lizardo (1):
Depend on GLib 2.20 rather than 2.22

Colin Walters (2):
[Makefile.am] Clean up CFLAGS handling, don’t override all: target
Add Tomeu’s prototype script for converting pygtk to pygi

Johan Dahlin (3):
Create overridden modules in two passes
Remove trailing whitespace
Pythonify. Avoid ; and () around if statements

John (J5) Palmieri (1):
override that wasn’t checked in – fixes some test cases

Olav Vitters (1):
Fix doap file

Simon van der Linden (18):
Initial import
Add PyGObject patches
Add a doap file
Update PyGObject patches
Remove PyGObject patches since they’ve been merged to master
Fix silent rules setup
Import pygtk properly to avoid failure on some setups
Use the right variable when looking up in sys.modules
Fix members initialization in metaclasses
Remove global checks for pointers and move them in type cases
that need them
sys.path must be modified after pygtk is imported
Suppress compilation warnings
Don’t set a default constructor for structures.
Initialize struct fields to 0 when allocating
Restore the overrides support
Remove support for pointers to basic types as input-only
argument and return value
Fix and complete overrides tests
Add modelines and copyright information to overrides modules

Tomeu Vizoso (28):
Add myself to pygi.doap
Treat GI_INFO_TYPE_INTERFACE same as GI_INFO_TYPE_OBJECT
Register interfaces
Add support for Any arguments
Add stuff to .gitignore
Accept 0 as a valid value for flag and enum arguments
Structs in arrays are not marshalled correctly
A few tests about interfaces
Use the limit constants from glib and interpret G_MAXUINT32 as
PyLong_FromLongLong
Revert “Use the limit constants from glib and interpret
G_MAXUINT32 as PyLong_FromLongLong”
Set a default constructor for boxed structs that don’t have one
Use the limit constants from glib and interpret G_MAXUINT32 as
PyLong_FromLongLong
The array field ‘length’ starts to count from the C arg list, so
need to decrement when it’s a method
Add Gtk.keysyms to overrides
Always create the .so link
Use GIMarshallingTests (old TestGI) in gobject-introspection
Add metadata to the .doap file
Add gdb and valgrind variants for the tests
Allow creating structs with pointers
Add support for foreign structs
Add examples/cairo-demo.py
Require PyCairo
Update to latest version of the pygi-convert.sh script
Implement vfuncs.
Add missing file to tarballs
Add one more missing file to tarballs
Add more stuff to the tarballs
One more missing file…

Zach Goldberg (6):
Implementation callback support with scoping and basic argument support.
Add Zach Goldberg as a pygi maintainer
Move some tests from test_gi to test_everything
Implement nullable argument support, including tests
Fix a typo in pygi-callbacks.c header
Bump version for release 0.5.0

PyGi Hackfest Day 4 – A ‘call back’ to the past

This morning on Day 4 of the Gnome/Python hackfest we had several major milestones. Among them includes

  • The PyGi test suite runs with only two out of 200 failures under Python 3
  • Callback support (including scoping and primitive argument/error support) landed in PyGi master
  • Working vfunc support, to be pushed soon
  • Working nullable argument support, to be pushed soon
  • We officially decided that PyGi is not to be pronounced as piggy, but instead to be sounded out as Pie-Gee-Aiiy (Py, G. I.).
  • PyGi will have a release this weekend numbered 0.5. We will use the even/odd numberings to indicate stable and unstable releases. The number 0.5 was chosen to show that we believe that PyGi is in a usable “Beta”-ish state (as opposed to a 0.1 which represents Alpha quality). We want to encourage people to use PyGi and help us find problems. We believe many will have a lot of success testing PyGi, however we know there is more work to be done.
  • PyGi 0.5 will depend on the to-be-released (this weekend) GObject Introspection 0.6.10
  • Inclusion of myself (Zach Goldberg) as an official maintainer of PyGi

Gnome/Python Hackfest 2010 – Day 2, or: How to actually coordinate packages.

So it’s day two of the PyGI hackfest.  You can find some good coverage of Day 1 at Colin Walter’s Blog and John (J5) Palmieri’s Blog. The first few hours of day two include:

  • Made the decision that PyGI will depend on the GObject-Introspection 0.6.10 (to be released). This is because PyGI needs some new testing modules exported by GI as well as newer (faster!) APIs.
  • Maemo will originally get GI 0.6.7 and later be bumped to 0.6.10.
  • PyGI Callbacks patches all merged and are going through make check and patch review now
  • David Malcom got PyGI to compile and (mostly) link to Python 3.0
  • Tomeu got his hard drive to work
  • Colin Walters has been slacking
  • J5 has been compiling code all morning
  • We lack music. 🙁