Git Submodules, External Repositories, and Deployment

Posted by Vincent Woo Tue, 26 Aug 2008 07:41:00 GMT

Even though I switched a few of my projects from Subversion to Git a couple of months ago, I still had lots to learn. So I dedicated last weekend to use submodules for Rails plugins. Unfortunately, it wasn’t the quick conversion I thought it would be but it turned out for the best.

Git Submodules

External repository references was handled so elegantly by Piston for Subversion repositories that I was willing to wait for its Git support. But I came across this blog post that dispelled misconceptions and it turns out submodules act just like Piston. In short, submodules will keep track of which revision it is on and won’t hit external resources unless it has to. Sweet! Submodules are great and simple to work with on the development side. Do: 
  git submodule add git://github.com/rails/acts_as_list.git vendor/plugins/acts_as_list
  git submodule init
  git submodule update

And you’re golden. But there is still much to do. In order to deploy efficiently with Capistrano, I’d needed to do a “set :deploy_via, :remote_cache”. Onwards in search of a Internet accessible Git repository that I can call my own.

Git Repository Hosting

Enter Railsplayground.com Ruby on Rails Hosting. They have been hosting this blog for several years and have always been great for shared hosting. Bonus points for free Subversion and Trac hosting, up to 1GB disk space, with any hosting account.

Little did I know that they have been doing Git repository hosting for the past several months. Their homepage doesn’t mention it although their forum as well as their sister site do.

If you’re thinking about their git repository hosting solution, there are a couple of things to keep in mind.

PRO: Custom control panel: easy to create/delete git repositories and manage user accounts that has access to them

PRO: Trac integration is underway

CON: The only protocols they support are HTTP and HTTPS. Update: Railsplayground will support the Git protocol very soon. See comments below.

CON: It requires creating a ~/.netrc with my login and password. I don’t want to store any passwords in plain text on my laptop so I symlinked the file to an encrypted vault where my project files are anyways. Update: Railsplayground will support the Git protocol over ssh very soon. See comments below.

MacPorts Pain

I’m using MacPorts on my Macbook Pro to compile git. This setup worked fine before but I couldn’t access the newly created git repository. 
  $ git push upload master
  fatal: git-push is not available for http/https repository when not compiled with USE_CURL_MULTI
  error: failed to push some refs to 'https://secure2.svnrepository.com/g_username/repository_name/'
Unfortunately, MacPorts added in a patch to remove the USE_CURL_MULTI flag for stability reasons. Since the bug is triggered by a very large checkout on a powerpc mac, not applicable in my case, I removed the patch by editing the git Portfile. 
  #/opt/local/var/macports/sources/rsync.macports.org/release/ports/devel/git-core/Portfile

  patchfiles   patch-Makefile.diff patch-http.h.diff  # Before
  patchfiles   patch-Makefile.diff  # After
I’m not sure how to tell MacPorts to recompile git. I cheapened out and added a variant to force a recompile. 
  sudo port deactivate git-core @1.6.0_1+doc+svn
  sudo port install git-core +doc +svn +bash_completion
Once that was resolved, I encountered yet another error. 
  $ git push upload master
  error: Cannot access URL https://secure2.svnrepository.com/username/repository_name/, return code 1
  error: failed to push some refs to 'https://secure2.svnrepository.com/username/repository_name/'
Turned out that curl needs to be compiled with the ssl variant. Normally, this should work: 
  $ sudo port install curl +ssl

  --->  Fetching curl-ca-bundle
  --->  Attempting to fetch certdata-1.48.txt from http://lxr.mozilla.org/seamonkey/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1
  --->  Verifying checksum(s) for curl-ca-bundle
  Error: Checksum (md5) mismatch for certdata-1.48.txt
  Error: Checksum (sha1) mismatch for certdata-1.48.txt
  Error: Checksum (rmd160) mismatch for certdata-1.48.txt
  Error: Target org.macports.checksum returned: Unable to verify file checksums
  Error: The following dependencies failed to build: curl-ca-bundle
  Error: Status 1 encountered during processing.
MacPorts should already have this fixed but just in case, I got around the error by skipping checksums. 
  sudo port install curl +ssl checksum.skip=yes

Deployment Issues

Here’s the git related entries in my Capistrano deployment script: 
  set :repository, "https://secure2.svnrepository.com/g_username/project_name"
  set :scm, 'git'
  set :deploy_via, :remote_cache
  set :branch, "master"
  set :git_shallow_clone, 1
  set :git_enable_submodules, 1
  default_environment['GIT_SSL_NO_VERIFY'] = 'true'
But deployment failed midway: 
  $ cap staging deploy
  Initialized empty Git repository in /home/username/project_name/shared/cached-copy/vendor/plugins/acts_as_list/.git/
  github.com[0: 65.74.177.129]: errno=Connection timed out
  ** fatal: unable to connect a socket (Connection timed out)
  ** Clone of 'git://github.com/rails/acts_as_list.git' into submodule path 'vendor/plugins/acts_as_list' failed
The server firewall was getting in the way so I punched a hole for the git protocol: 
  iptables -I OUTPUT -p tcp --dport 9418 -j ACCEPT

Sweet Oblivion

With that all out of the way, I can now successfully deploy to the server the way I want to. Now I can get back to work :)

Posted in ,  | Tags , , , , , , , , , , ,  | 4 comments | no trackbacks

Automating Credentials in Capistrano

Posted by Vincent Woo Wed, 13 Jun 2007 20:40:00 GMT

Here’s a small tip that’s working well for me. If the user and password variables are set in the capistrano deployment scripts, then capistrano won’t prompt for the credentials anymore. It’s best to store them in ~/.caprc so you don’t commit your credentials to your SCM for all to see.

I’ve accumulated several different sets of credentials now that I’ve been working on several rails projects with different teams with their own login naming conventions. The simple assignment in .caprc just won’t do anymore.

Well, .caprc is just ruby code. Additionally, proc’s can be assigned to values to defer code execution until the value is first accessed. That way, the proc will have access to the application value in it’s local scope. Then we could do something like the following in .caprc:

set :user, Proc.new {
                case application
                   when 'project1': 'user1'
                   when 'project2': 'user2'
                 end
}

set :password, Proc.new {
                 case application
                   when 'project1': 'pass1'
                   when 'project2': 'pass2'
                 end
}

set :svn_username, Proc.new {
                 case application
                   when 'project1': 'svn_user1'
                   when 'project2': 'svn_user2'
                 end
}

Posted in  | Tags , , ,  | no comments | no trackbacks

Updated Cap Cheatsheet

Posted by Vincent Woo Thu, 17 Aug 2006 03:09:00 GMT

Went a little further with the Capistrano setup and realized that I completely ignored mongrel and its cluster plugin. So after a few deletions here and some revising there, an updated cheatsheet is now ready for your downloading pleasure.

.png .pdf

Posted in  | Tags , , ,  | no comments | no trackbacks

Capistrano Cheatsheet

Posted by Vincent Woo Tue, 15 Aug 2006 04:31:00 GMT

I recently looked into using Capistrano for deploying webapps to the server. I needed to extend some of the tasks since I won’t be using fcgi so I spent the time mapping out how the tasks relate to one another. Let me know if there are errors for me to correct.


.png .pdf

Posted in  | Tags ,  | no comments | no trackbacks