Terminal multiplexers like GNU Screen and tmux are indispensable for running remote jobs, but they don’t always play nicely with X11 forwarding (i.e. ssh -X or ssh -Y). This can result in remotely-launched GUI programs failing to display and similar difficulties. Typically, I’ll start a job in screen while at lab, go home, and reconnect remotely, only to find that X11 forwarding works correctly outside of but not inside of screen. The same happens with tmux.

Most aggravating to me in this situation is the failure of a bash alias cpd that I use quite frequently:

1

alias cpd='echo -en `pwd`/ | xclip -selection clipboard'

This alias copies the current working directory into the clipboard, making it easier to navigate the filesystem, copy files between machines, and so on. Unfortunately, it requires X11 forwarding to work.

The root cause of the problem here is that the DISPLAY environment variable needs to be set appropriately or X11 forwarding breaks. For example, when running locally DISPLAY is usually :0, but when logged in remotely it should be localhost:x.0, where x is some number. Normally you don’t have to think about this, but when reconnecting to a screen session the terminal doesn’t know when DISPLAY needs to change. Setting DISPLAY manually is possible but entirely too much work to do on a regular basis.

A solution …

To resolve this problem in a way that is transparent to the user, we can automatically cache the value of DISPLAY before every command run outside of screen, and set the value of DISPLAY to the cache before every command run inside of screen.

You can paste this into your .bashrc. Beware the potential complication if you use PROMPT_COMMAND.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# -- Improved X11 forwarding through GNU Screen (or tmux).
# If not in screen or tmux, update the DISPLAY cache.
# If we are, update the value of DISPLAY to be that in the cache.
function update-x11-forwarding
{
    if [ -z "$STY" -a -z "$TMUX" ]; then
        echo $DISPLAY > ~/.display.txt
    else
        export DISPLAY=`cat ~/.display.txt`
    fi
}

# This is run before every command.
preexec() {
    # Don't cause a preexec for PROMPT_COMMAND.
    # Beware!  This fails if PROMPT_COMMAND is a string containing more than one command.
    [ "$BASH_COMMAND" = "$PROMPT_COMMAND" ] && return 

    update-x11-forwarding

    # Debugging.
    #echo DISPLAY = $DISPLAY, display.txt = `cat ~/.display.txt`, STY = $STY, TMUX = $TMUX  
}
trap 'preexec' DEBUG

… that mostly works

This can break down when you are doing more complicated things. For example:

• Create a screen session on a work machine.
• Go home.
• In terminal A, ssh in and re-attach.
• In terminal B, ssh in again.
• Disconnect terminal B.

The screen session running at work, visible in terminal A, ends up with a bad DISPLAY setting and X11 forwarding fails. Fortunately, it is easy to recover from these kinds of failures: Just detach and re-attach the screen session.

A note on security

If you’re using X11 forwarding, be aware there are security risks involved. It’s best to use X11 forwarding selectively, either with ssh -X on the terminal or by specifically configuring your ~/.ssh/config file to allow it for only certain machines. For example, this appears in my ssh config file for a remote machine capek that I trust. All other connections have X11 forwarding disabled by default.

1
2
3
4
5
6
7
8
9

Host capek
     Hostname capek.stanford.edu
     User teichman
     ForwardX11 yes
     ForwardX11Trusted yes

Host *
     ForwardX11 no
     ForwardAgent no

Terminal multiplexers like GNU Screen and tmux are indispensable for running remote jobs, but they don’t always play nicely with X11 forwarding (i.e. ssh -X or ssh -Y). This can result in remotely-launched GUI programs failing to display and similar difficulties. Typically, I’ll start a job in screen while at lab, go home, and reconnect remotely, only to find that X11 forwarding works correctly outside of but not inside of screen. The same happens with tmux.

Most aggravating to me in this situation is the failure of a bash alias cpd that I use quite frequently:

1

alias cpd='echo -en `pwd`/ | xclip -selection clipboard'

This alias copies the current working directory into the clipboard, making it easier to navigate the filesystem, copy files between machines, and so on. Unfortunately, it requires X11 forwarding to work.

The root cause of the problem here is that the DISPLAY environment variable needs to be set appropriately or X11 forwarding breaks. For example, when running locally DISPLAY is usually :0, but when logged in remotely it should be localhost:x.0, where x is some number. Normally you don’t have to think about this, but when reconnecting to a screen session the terminal doesn’t know when DISPLAY needs to change. Setting DISPLAY manually is possible but entirely too much work to do on a regular basis.

A solution …

To resolve this problem in a way that is transparent to the user, we can automatically cache the value of DISPLAY before every command run outside of screen, and set the value of DISPLAY to the cache before every command run inside of screen.

You can paste this into your .bashrc. Beware the potential complication if you use PROMPT_COMMAND.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# -- Improved X11 forwarding through GNU Screen (or tmux).
# If not in screen or tmux, update the DISPLAY cache.
# If we are, update the value of DISPLAY to be that in the cache.
function update-x11-forwarding
{
    if [ -z "$STY" -a -z "$TMUX" ]; then
        echo $DISPLAY > ~/.display.txt
    else
        export DISPLAY=`cat ~/.display.txt`
    fi
}

# This is run before every command.
preexec() {
    # Don't cause a preexec for PROMPT_COMMAND.
    # Beware!  This fails if PROMPT_COMMAND is a string containing more than one command.
    [ "$BASH_COMMAND" = "$PROMPT_COMMAND" ] && return 

    update-x11-forwarding

    # Debugging.
    #echo DISPLAY = $DISPLAY, display.txt = `cat ~/.display.txt`, STY = $STY, TMUX = $TMUX  
}
trap 'preexec' DEBUG

… that mostly works

This can break down when you are doing more complicated things. For example:

• Create a screen session on a work machine.
• Go home.
• In terminal A, ssh in and re-attach.
• In terminal B, ssh in again.
• Disconnect terminal B.

The screen session running at work, visible in terminal A, ends up with a bad DISPLAY setting and X11 forwarding fails. Fortunately, it is easy to recover from these kinds of failures: Just detach and re-attach the screen session.

A note on security

If you’re using X11 forwarding, be aware there are security risks involved. It’s best to use X11 forwarding selectively, either with ssh -X on the terminal or by specifically configuring your ~/.ssh/config file to allow it for only certain machines. For example, this appears in my ssh config file for a remote machine capek that I trust. All other connections have X11 forwarding disabled by default.

1
2
3
4
5
6
7
8
9

Host capek
     Hostname capek.stanford.edu
     User teichman
     ForwardX11 yes
     ForwardX11Trusted yes

Host *
     ForwardX11 no
     ForwardAgent no

The tools grep, ack, and ag are great for searching repositories, but often we don’t just want to find a match, we want to navigate to a match and make changes. What follows are some bash functions for rapidly searching a repository and opening selected matches.

This is my primary method for navigating source repositories. I’ve written enough emails with this content that I decided it was time to write it up properly once and for all. I’ll discuss C++ in particular here, but the functions are easy to adapt to whatever language you work in.

cgrep and yank

Here’s a list of the most frequently used commands in my Bash history, with a count in the left column.

1
2
3
4
5
6
7

17959 l
11922 cd
10079 make
4586 git
3986 hg
3405 cgrep      <------------
2874 cat

So what is this cgrep thing, and why is it my 6th-most-used command? It greps through all C++ source files and returns matches like this:

This is similar to the output of ack --nogroup --cpp imread, but with the addition that you can quickly open one of the matches and navigate to the appropriate line number using yank.

1

$ cgrep imread | yank 6

Running this will open ./src/program/image_cut.cpp in a running Emacs window and move to the imread call on line 253. I use emacsclient to make this possible, and presumably you can do something similar with Vim or whatever editor you use.

It looks like this.

More examples from my Bash history

Because this is all happening on the command line, you can apply whatever Unix filters might be useful. Here I wanted to see all usages of isinf except those in the PCL library.

1

$ cgrep isinf | grep -v './pcl_trunk/'`

Because cgrep is using grep under the hood, all the usual options you know and love still apply. Here I was showing usages of the Eigen library’s InnerIterator, with 10 following lines of context. This makes it easy to scan for errors, then jump to the location with yank to fix them.

1

cgrep InnerIterator -A10

We can also define a variation on cgrep to only search in header files. Here I was looking for compound assignment operators in class declarations.

1

hgrep 'operator[^=!]='`

Details

Here’s how cgrep and friends are implemented. You can paste this in your .bashrc. To hook up yank to emacsclient, you’ll also probably want export EDITOR='emacsclient -n' in your .bashrc and (server-start) in your Emacs init file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function cfiles {
    find -regextype posix-egrep -regex '.*\.h$|.*\.hpp$|.*\.c$|.*\.cpp|.*\.cc$'
}

function cgrep {
   2>/dev/null grep -n "$@" $(cfiles) | nl | grep "$@"
}

function hfiles {
    find -regextype posix-egrep -regex '.*\.h|.*\.hpp'
}

function hgrep {
    2>/dev/null grep -n "$@" $(hfiles) | nl | grep "$@"
}

function pyfiles {
    find -regextype posix-egrep -regex '.*\.py$' | grep -v '#'
}

function pygrep {
   2>/dev/null grep -n "$@" $(pyfiles) | nl | grep "$@"
}

function yank {
    grep "^\s*$@\s" | awk '{print $2}' | sed 's/-\([0-9]*\)-/:\1:/g' | awk -F: '{print $1 " +" $2}' | awk '{print $2 " " $1}' | xargs ${EDITOR:?EDITOR must be set.}
}

Related, the cfiles function is handy for doing large search and replace operations.

1

sed -i 's/NameMapping2/NameMapping/g' `cfiles`

If you’re wondering how to easily get a list of your own most frequently used commands, you can use something like this. The awk command likely won’t translate to your machine because I am using a custom definition of HISTTIMEFORMAT, but you get the idea.

1

$ history | awk '{print $6}' | sort | uniq -c | sort -n

Changelog

• 2013-12-10: Added screencast example, clarified that the point is fast navigation rather than just search.
• 2013-12-13: Made video scale automatically.

The tools grep, ack, and ag are great for searching repositories, but often we don’t just want to find a match, we want to navigate to a match and make changes. What follows are some bash functions for rapidly searching a repository and opening selected matches.

This is my primary method for navigating source repositories. I’ve written enough emails with this content that I decided it was time to write it up properly once and for all. I’ll discuss C++ in particular here, but the functions are easy to adapt to whatever language you work in.

cgrep and yank

Here’s a list of the most frequently used commands in my Bash history, with a count in the left column.

1
2
3
4
5
6
7

17959 l
11922 cd
10079 make
4586 git
3986 hg
3405 cgrep      <------------
2874 cat

So what is this cgrep thing, and why is it my 6th-most-used command? It greps through all C++ source files and returns matches like this:

This is similar to the output of ack --nogroup --cpp imread, but with the addition that you can quickly open one of the matches and navigate to the appropriate line number using yank.

1

$ cgrep imread | yank 6

Running this will open ./src/program/image_cut.cpp in a running Emacs window and move to the imread call on line 253. I use emacsclient to make this possible, and presumably you can do something similar with Vim or whatever editor you use.

It looks like this.

More examples from my Bash history

Because this is all happening on the command line, you can apply whatever Unix filters might be useful. Here I wanted to see all usages of isinf except those in the PCL library.

1

$ cgrep isinf | grep -v './pcl_trunk/'`

Because cgrep is using grep under the hood, all the usual options you know and love still apply. Here I was showing usages of the Eigen library’s InnerIterator, with 10 following lines of context. This makes it easy to scan for errors, then jump to the location with yank to fix them.

1

cgrep InnerIterator -A10

We can also define a variation on cgrep to only search in header files. Here I was looking for compound assignment operators in class declarations.

1

hgrep 'operator[^=!]='`

Details

Here’s how cgrep and friends are implemented. You can paste this in your .bashrc. To hook up yank to emacsclient, you’ll also probably want export EDITOR='emacsclient -n' in your .bashrc and (server-start) in your Emacs init file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function cfiles {
    find -regextype posix-egrep -regex '.*\.h$|.*\.hpp$|.*\.c$|.*\.cpp|.*\.cc$'
}

function cgrep {
   2>/dev/null grep -n "$@" $(cfiles) | nl | grep "$@"
}

function hfiles {
    find -regextype posix-egrep -regex '.*\.h|.*\.hpp'
}

function hgrep {
    2>/dev/null grep -n "$@" $(hfiles) | nl | grep "$@"
}

function pyfiles {
    find -regextype posix-egrep -regex '.*\.py$' | grep -v '#'
}

function pygrep {
   2>/dev/null grep -n "$@" $(pyfiles) | nl | grep "$@"
}

function yank {
    grep "^\s*$@\s" | awk '{print $2}' | sed 's/-\([0-9]*\)-/:\1:/g' | awk -F: '{print $1 " +" $2}' | awk '{print $2 " " $1}' | xargs ${EDITOR:?EDITOR must be set.}
}

Related, the cfiles function is handy for doing large search and replace operations.

1

sed -i 's/NameMapping2/NameMapping/g' `cfiles`

If you’re wondering how to easily get a list of your own most frequently used commands, you can use something like this. The awk command likely won’t translate to your machine because I am using a custom definition of HISTTIMEFORMAT, but you get the idea.

1

$ history | awk '{print $6}' | sort | uniq -c | sort -n

Changelog

• 2013-12-10: Added screencast example, clarified that the point is fast navigation rather than just search.
• 2013-12-13: Made video scale automatically.

Unsurprisingly, gross physical deformation of an RGBD sensor body will produce changes in the distortion pattern of the sensor. Surprisingly, you can cause this yourself quite easily without being aware of it.

We ran in to this when mounting an Asus Xtion Pro Live using velcro cinch straps around the sensor body. Pulling to hand tightness easily causes noticeable changes in the distortion pattern. If you’re trying to calibrate a sensor using our CLAMS work, this can really mess you up.

Here are some examples showing the change in the raw sensor output when tightening vs loosening the straps. Mount your sensors with care!

Unsurprisingly, gross physical deformation of an RGBD sensor body will produce changes in the distortion pattern of the sensor. Surprisingly, you can cause this yourself quite easily without being aware of it.

We ran in to this when mounting an Asus Xtion Pro Live using velcro cinch straps around the sensor body. Pulling to hand tightness easily causes noticeable changes in the distortion pattern. If you’re trying to calibrate a sensor using our CLAMS work, this can really mess you up.

Here are some examples showing the change in the raw sensor output when tightening vs loosening the straps. Mount your sensors with care!

After presenting our work on unsupervised calibration of PrimeSense sensors at RSS2013, several people asked if the distortion is a function of view angle. In light of how these sensors work this is a reasonable concern. However, we’re not seeing evidence of this. If you are, please contact me.

What follows are animated gifs of the learned distortion model working at different view angles. If the depth distortion was significantly a function of view angle, I’d expect the results to appear unreasonable, and this isn’t the case.

Here’s a typical scene looking at a long table, wooden chairs, and a white wall. At 7 meters, the wall is highly distorted in the raw data, and applying the learned distortion model results in a reasonably flat wall.

Switching to a top-down view, here’s the result at about 40 degrees. Again, the wall becomes flat after applying the learned distortion model.

At about 20 degrees, the distortion is getting harder to see from top-down. This is because the distortion is a depth multiplier; at this angle the displacement of the points is mostly in line with the wall.

Viewing the same scene from the side is telling. We should be seeing straight lines at the intersection of the flat wall and the left & top edges of the view frustum. These straight lines only appear after applying the learned distortion model.

This corresponds nicely to the learned distortion model, which says the upper left corner peels away from the sensor. (Red means raw data must be pushed further away to produce accurate readings, blue means raw data must be pulled closer.)

After presenting our work on unsupervised calibration of PrimeSense sensors at RSS2013, several people asked if the distortion is a function of view angle. In light of how these sensors work this is a reasonable concern. However, we’re not seeing evidence of this. If you are, please contact me.

What follows are animated gifs of the learned distortion model working at different view angles. If the depth distortion was significantly a function of view angle, I’d expect the results to appear unreasonable, and this isn’t the case.

Here’s a typical scene looking at a long table, wooden chairs, and a white wall. At 7 meters, the wall is highly distorted in the raw data, and applying the learned distortion model results in a reasonably flat wall.

Switching to a top-down view, here’s the result at about 40 degrees. Again, the wall becomes flat after applying the learned distortion model.

At about 20 degrees, the distortion is getting harder to see from top-down. This is because the distortion is a depth multiplier; at this angle the displacement of the points is mostly in line with the wall.

Viewing the same scene from the side is telling. We should be seeing straight lines at the intersection of the flat wall and the left & top edges of the view frustum. These straight lines only appear after applying the learned distortion model.

This corresponds nicely to the learned distortion model, which says the upper left corner peels away from the sensor. (Red means raw data must be pushed further away to produce accurate readings, blue means raw data must be pulled closer.)

This is a test post to see if Octopress is suitable for my purposes. Conclusion: yes. There’s nothing here for general consumption, but I’ve left this page up in the offchance you’re wondering if Octopress is for you.

YouTube videos

You can more or less just paste the YouTube embed line directly into the markdown file. I needed to add ‘http:’ to the URL supplied by YouTube. Adding ?rel=0&vq=hd1080 to the end of the URL will prevent it from showing related video suggestions at the end and make it play in HD.

Images

Images should be placed in the octopress/source/images directory. A link path looks like /images/2013-07-15_oblique_angle_test/00-animated.gif. When the site is generated, they’ll end up in the octopress/public/images directory.

Math

Latex is not supported by default but it is quite easy to add:

Terminal output

1
2
3
4
5
6
7
8
9
10
11
12
13
14

teichman@aluminum:~/octopress$ ls -l
total 60
-rw-rw-r--  1 teichman teichman  1303 Jul 15 17:55 CHANGELOG.markdown
-rw-rw-r--  1 teichman teichman   413 Jul 15 17:55 config.rb
-rw-rw-r--  1 teichman teichman   643 Jul 15 17:55 config.ru
-rw-rw-r--  1 teichman teichman  2865 Jul 15 18:06 _config.yml
-rw-rw-r--  1 teichman teichman   450 Jul 15 17:55 Gemfile
-rw-rw-r--  1 teichman teichman  1412 Jul 15 17:57 Gemfile.lock
drwxrwxr-x  2 teichman teichman  4096 Jul 15 17:55 plugins
drwxrwxr-x  7 teichman teichman  4096 Jul 15 18:09 public
-rw-rw-r--  1 teichman teichman 15660 Jul 15 17:55 Rakefile
-rw-rw-r--  1 teichman teichman  2839 Jul 15 17:55 README.markdown
drwxrwxr-x  6 teichman teichman  4096 Jul 15 18:01 sass
drwxrwxr-x 10 teichman teichman  4096 Jul 15 18:04 source

Code

1
2
3

function cgrep {
  2>/dev/null grep -n "$@" $(cfiles) | nl | grep "$@"
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

class Trajectory : public Serializable
{
public:
  Trajectory() {}
  //! Deep copy
  Trajectory(const Trajectory& other);
  //! Deep copy
  Trajectory& operator=(const Trajectory& other);
  ~Trajectory();

  void clear();
  //! Destroys everything inside.
  void resize(size_t num);
  void set(size_t idx, const Eigen::Affine3d& transform);
  const Eigen::Affine3d& get(size_t idx) const;
  bool exists(size_t idx) const;
  void remove(size_t idx);
  size_t size() const { return transforms_.size(); }
  size_t numValid() const;
  std::string status(const std::string& prefix) const;

  void serialize(std::ostream& out) const;
  void deserialize(std::istream& in);

protected:
  std::vector<Eigen::Affine3d*> transforms_;
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

#!/usr/bin/python

import os, sys, string, argparse, smtplib
from email.mime.text import MIMEText

# -- Parse args.
parser = argparse.ArgumentParser()
parser.add_argument('--fr', nargs='+')
parser.add_argument('--to', nargs='+')
parser.add_argument('--subj', nargs='+')
parser.add_argument('--body', nargs='*')
args = parser.parse_args()

to = ', '.join(args.to)
subj = ' '.join(args.subj)
if args.body is None:
    body = ''
else:
    body = ' '.join(args.body)

print 'To:   ' + to
print 'Subj: ' + subj
print 'Body: ' + body

# -- Set up the message.
msg = MIMEText(body)
msg['Subject'] = subj
msg['To'] = to

stanford = True
if stanford:
    server = smtplib.SMTP('smtp-unencrypted.stanford.edu')
    username = args.fr
else:
    # -- Connect to hard-coded gmail account.  Gross.
    username = 'anon.aoeu@gmail.com'
    password = '<YOUR PASSWORD>'
    server = smtplib.SMTP('smtp.gmail.com:587')
    server.starttls()
    server.login(username, password)

# -- Send.
fail = server.sendmail(username, args.to, msg.as_string())
if fail:
    print fail
else:
    print 'Email sent.'

server.quit()

Lists

The indentation isn’t quite showing up correctly in my browser, but this can probably be fixed.

• This is a list item
• Something else
  • Foo
  • Bar

Long text

Bacon ipsum dolor sit amet sirloin corned beef jowl prosciutto flank kielbasa, fatback salami bacon shankle hamburger pork chop tongue beef. Ham hock strip steak hamburger jowl, shoulder bacon fatback. Biltong meatloaf prosciutto bacon drumstick ground round venison swine cow jerky tri-tip tongue fatback rump. Tenderloin ribeye venison pork belly shankle. Short loin ham hock tenderloin strip steak leberkas jowl pancetta swine drumstick chuck.

Turkey filet mignon hamburger ball tip brisket beef ribs corned beef tenderloin andouille capicola kielbasa. Venison shoulder hamburger, capicola swine drumstick short ribs beef ribs pig corned beef pork pork loin meatloaf. Salami andouille meatball, biltong jerky bacon ball tip. Doner shoulder ground round tri-tip turducken chicken t-bone pork shank. Bresaola pig ribeye strip steak ham hock brisket boudin flank, bacon short ribs pancetta turducken biltong meatball rump. Kielbasa pastrami cow, meatloaf bacon short ribs tail tongue swine andouille rump ground round.

Short loin fatback jerky hamburger ribeye bacon swine bresaola. Corned beef shankle pork chop, pork fatback leberkas salami chuck ribeye sausage rump bresaola meatball tri-tip capicola. Hamburger chuck tongue capicola turkey ground round short loin pork meatball andouille tail cow. Pork loin sirloin frankfurter, turducken pastrami andouille short ribs tongue shank spare ribs flank. Chicken flank tenderloin short loin capicola sausage bresaola venison boudin prosciutto swine cow jerky. Tongue biltong boudin pig fatback, ribeye beef turducken. Tongue shank pork chop sirloin jowl.

Swine ham hock ribeye, meatball tri-tip salami ham pork pastrami tail beef. Drumstick bacon prosciutto ham hock pork belly beef, strip steak pork. Boudin strip steak flank tri-tip brisket ribeye ham. Pancetta meatball short loin tri-tip hamburger swine filet mignon pig kielbasa ribeye.

Jowl swine shankle turkey ham turducken pastrami prosciutto short ribs ground round beef ribs pig. Pancetta bacon fatback boudin kielbasa turducken. Sausage t-bone prosciutto jerky bresaola ground round leberkas jowl meatball capicola chuck. Boudin jerky bresaola rump pork loin prosciutto sirloin swine drumstick.

Thanks to Bacon Ipsum.