The title of this post may make it sound like it’s going to be about how git changed the face of software development and entered us into a
new era of coding collaboration. No. This post is about how to copy a subtree from one git repository to another, while keeping its history.
Let’s say you’ve been working on a component in an incubator-style repository, and it’s time to move it to the main project’s repository.
Simply copying the code would destroy the valuable history in the target repo, so that should be avoided. There are also lots of other
components in the incubator repo that you don’t want to move.
Step 1: Export your component to a temporary repository
You can do this using git filer-branch. Warning: this will reduce your local copy of the incubator repository to just your component.
Either clone it locally to a different path or be prepared to clone it again from the remote.
Now your repository has been reduced to just the contents and history of components/my-component.
Step 2: Create a patch for the entire history of your component
Use git format-patch to export all of your commits to a patch file. Props to @rombert for this idea.
incubator$ git format-patch --stdout --root $(git rev-list HEAD | tail -n 1) HEAD > my-component.patch
I feel like this command could use some explaining. First off, we’re telling format-patch to output everything to --stdout. Otherwise,
it would create one patch file per commit, which can get pretty clumsy if there are lots of them. Second, we’re passing the output of
git rev-list HEAD | tail -n 1 for the --root parameter. The enclosed command will find find the sha1 of the very first commit, while
the --root parameter will tell format-patch to include that commit, not start from it. Lastly, HEAD is the target ref, which is
basically the most recent commit.
Step 3: Apply the patch
Now it’s time to add your component to the main project. Of course, you don’t want to add your component to the root of the repository
(which is what all the paths in the patch are relative to). Thankfully, git supports prepending a path to all filenames in a patch.
main-project$ git am --directory components/my-component my-component.patch
Now your main project contains your component and all of its history. Some notes:
Only one branch will be copied. If your component development happened on multiple branches, that will be lost.
No tags will be copied to the main repository, you will need to tag your commits manually.
My parents recently asked me to install an IP video surveillance camera at their
house (for reasons). Since I already have a Linux server there running 24/7, it
was only natural that I somehow set it up to record the camera’s video feed (mp4
over RTSP). After a bit of googling I found ZoneMinder, which looked like
it did everything I needed. However, after wasting half a day fiddling with it,
I realized that it is, although very complex and feature-rich, not up to my
needs: no matter how you configure it, it will convert the video feed into
thousands of JPEG files and store them randomly on the filesystem. It also uses
a MySQL database to store its information (so extra dependencies), viewing the
camera live is very slow (refreshes a static image every 4-5 seconds), replaying
a recording needs the Java plug-in in your browser (which I don’t have on
Ubuntu) and you have to “export” a movie clip (i.e. it has to convert all the
images it saved back to a video). What a waste, but I can see how this solution
could be the most compatible with all possible environments (camera types, OSes,
etcetera). I was unable to set up authentication or a maximum total file size,
Being fed up with ZoneMinder, I ditched it and decided to try and hack together
something of my own. After fiddling for the next half of the day with ffmpeg,
I came up with a viable solution, in bash.
Unfortunately, I was not able to just copy the codec, which would have been
optimal in terms of resource (CPU/memory) usage on my server, but it doesn’t
use up that much. Now I can set OUTPUT_DIR to somewhere in my /var/www,
set up authentication on it via .htaccess and be done with it. Dad can now
easily view the recordings using his browser, and if he really needs to view the
live stream I can just bookmark the RTSP stream in VLC for him or something.
Welcome to the tutorial on how to successfully fuck up with find, the command line tool. In case you’re not familiar with the find command, RTFM, it’s freaking beautiful. However, in the hands of a non-RTFM-ing user (like me) it can be quite destructive. Let me demonstrate.
I wanted to use find to delete a bunch of files from a hierarchy, based on a simple name rule. So I thought I should use xargs in combination with find to delete them. After a quick lookup of the -print0 and -name arguments of the find command, I concluded that it would be a good idea to run the following command:
$ find . -print0 -name example | xargs -0 rm
Can you guess what it did? Let me help you: it deleted all the files under .. The directories are still there, but they’re not any help, are they? Now, fortunately for me, this was in a git repository that I had just pushed to a remote, so I was able to just clone it again.
In order to understand what happened, I had to actually read the man page carefully and pay attention. Apparently, everything that comes after the path (in my case .) on the command line is treated as an expression by find, where every argument evaluates to true or false, and is ORed with the next one. The -print0 argument always returns true, because it’s only meant to change the output, not the actual filtering. Because of this, my -name argument was completely ignored.
To conclude, the correct command would have been:
$ find . -name example -print0 | xargs -0 rm
This way, the -name argument will take precedence over print0. However, because I actually spent time reading the manual this time, I discovered find also has a -delete argument, so xargs is not actually needed at all:
Given how developer friendly as Ruby on Rails is, getting a development environment up and running can be surprisingly tricky. I will describe the process that I consider to be “best practice” here, step by step. Although my guide will be focused on Ubuntu, the general idea should be the same on other distros as well. Also, please take note of the date of this post. If you’re reading this even a few months after I wrote this, don’t be surprised if your mileage will vary.
First off, we’re going to need Ruby, right? So let’s install that, and also throw in curl (we’re going to need it in a bit):
Update: actually, you don’t need Ruby installed on your system at all. Installing curl is enough here:
$ sudo apt-get install curl
Next, we’re going to install RVM. RVM is, at the website puts it, a Ruby Version Manager. Basically, it allows you to install multiple versions of Ruby in your home folder and use them seamlessly (as if they were the default Ruby in the system). This is useful for us because the version of Ruby Ubuntu installed is 1.8.7 (as of this writing), while the latest stable version of Ruby is 1.9.3. RVM also allows us to keep separate collections of gems (called gemsets, for obvious reasons). This is very useful when working with multiple projects that may have conflicting dependencies.
Installing RVM is dead simple. Just run the command under Quick Install:
$ curl -L get.rvm.io | bash -s stable
Warning: the above command might not be the latest one. Make sure to double check with the RVM website.
Now, close the current terminal and open a new one. This is so that RVM can load into the new bash session. Now, let’s see what RVM suggests we should install:
$ rvm requirements
Requirements for Linux ( DISTRIB_ID=UbuntuDISTRIB_RELEASE=11.10DISTRIB_CODENAME=oneiricDISTRIB_DESCRIPTION="Ubuntu 11.10" )NOTE: 'ruby' represents Matz's Ruby Interpreter (MRI) (1.8.X, 1.9.X) This is the *original* / standard Ruby Language Interpreter 'ree' represents Ruby Enterprise Edition 'rbx' represents Rubiniusbash >= 4.1 requiredcurl is requiredgit is required (>= 1.7 for ruby-head)patch is required (for 1.8 rubies and some ruby-head's).To install rbx and/or Ruby 1.9 head (MRI) (eg. 1.9.2-head),then you must install and use rvm 1.8.7 first.Additional Dependencies:# For Ruby / Ruby HEAD (MRI, Rubinius, & REE), install the following:
ruby: /usr/bin/apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion# For JRuby, install the following:
jruby: /usr/bin/apt-get install curl g++ openjdk-6-jre-headless jruby-head: /usr/bin/apt-get install ant openjdk-6-jdk# For IronRuby, install the following:
ironruby: /usr/bin/apt-get install curl mono-2.0-devel
Install the dependencies suggested by RVM for Ruby (see the marked lines above and make sure to use sudo).
Good. Now it’s time to install Ruby! Again. The next command will install the latest stable Ruby into your home directory (so the system is not affected). Beware, it will take a long time to complete, as it downloads the Ruby source code and compiles it!
$ rvm install ruby
Now, if the command above finishes with a message similar to:
RVM is not a function, selecting rubies with 'rvm use ...' will not work.
Please visit https://rvm.io/integration/gnome-terminal/ for a solution.
Then do visit that URL and follow the instructions there. Finished? Good. Open a new terminal, again. Now type:
$ rvm use ruby
If all went well, you should see a green message telling you which Ruby is being used. Next, you should create a gemset (we talked about them in the beginning) for your new project, and also switch to it:
$ rvm gemset create myproject && rvm gemset use myproject
You finally have your environment set up. Next step is to install the rails gem:
$ gem install rails --no-rdoc --no-ri
Note: the —no-rdoc —no-ri parameters are passed so that it doesn’t waste time installing docs that are available on the Internet anyway. If you wish to have them installed for some reason, just omit those parameters.
Good, now you’re ready to create your new Rails app! Simply type:
$ rails new myproject
To run the new app:
$ rails server
You just got an error now, didn’t you? Did it sound like this?
Now, in order to actually install that gem, just run:
in your project’s directory. The bundle command basically looks in your Gemfile and installs / updates dependencies based on it. Running the app should now actually finally ultimately work:
$ rails server
=> Booting WEBrick=> Rails 3.2.3 application starting in development on http://0.0.0.0:3000=> Call with -d to detach=> Ctrl-C to shutdown server[2012-04-17 00:24:51] INFO WEBrick 1.3.1[2012-04-17 00:24:51] INFO ruby 1.9.3 (2012-02-16) [x86_64-linux][2012-04-17 00:24:51] INFO WEBrick::HTTPServer#start: pid=4595 port=3000
Now, every time you open a new terminal and intend to run the project, you will have to:
$ rvm use ruby
$ rvm gemset use myproject
$ rails server
If you want to skip the two rvm commands, you can use an .rvmrc file. That, however, is beyond the scope of this post, and is documented here.
As I was saying, surprisingly tricky. Sure, you could have used the Ruby and Rails provided by your package manager, but given the rapid development of both, you would be several major versions behind and lacking many features. Also, managing multiple projects is now a breeze. Just install the required Ruby (some projects might rely on older versions of it), create a new gemset for it, install dependencies, run.