Git Submodules, External Repositories, and Deployment
Even though I switched a few of my projects from Subversion to Git a couple of months ago, I still had lots to learn. So I dedicated last weekend to use submodules for Rails plugins. Unfortunately, it wasn't the quick conversion I thought it would be but it turned out for the best.
Git Submodules
External repository references was handled so elegantly by Piston for Subversion repositories that I was willing to wait for its Git support. But I came across this blog post that dispelled misconceptions and it turns out submodules act just like Piston. In short, submodules will keep track of which revision it is on and won't hit external resources unless it has to. Sweet!
Submodules are great and simple to work with on the development side. Do:
git submodule add git://github.com/rails/acts_as_list.git vendor/plugins/acts_as_list
git submodule init
git submodule update
And you're golden. But there is still much to do. In order to deploy efficiently with Capistrano, I'd needed to do a "set :deploy_via, :remote_cache". Onwards in search of a Internet accessible Git repository that I can call my own.
Git Repository Hosting
Enter Railsplayground.com Ruby on Rails Hosting. They have been hosting this blog for several years and have always been great for shared hosting. Bonus points for free Subversion and Trac hosting, up to 1GB disk space, with any hosting account.
Little did I know that they have been doing Git repository hosting for the past several months. Their homepage doesn't mention it although their forum as well as their sister site do.
If you're thinking about their git repository hosting solution, there are a couple of things to keep in mind.
PRO: Custom control panel: easy to create/delete git repositories and manage user accounts that has access to them
PRO: Trac integration is underway
CON: The only protocols they support are HTTP and HTTPS. Update: Railsplayground will support the Git protocol very soon. See comments below.
CON: It requires creating a ~/.netrc with my login and password. I don't want to store any passwords in plain text on my laptop so I symlinked the file to an encrypted vault where my project files are anyways. Update: Railsplayground will support the Git protocol over ssh very soon. See comments below.
MacPorts Pain
I'm using MacPorts on my Macbook Pro to compile git. This setup worked fine before but I couldn't access the newly created git repository.
$ git push upload master
fatal: git-push is not available for http/https repository when not compiled with USE_CURL_MULTI
error: failed to push some refs to 'https://secure2.svnrepository.com/g_username/repository_name/'
Unfortunately, MacPorts added in a patch to remove the USE_CURL_MULTI flag for stability reasons. Since the bug is triggered by a very large checkout on a powerpc mac, not applicable in my case, I removed the patch by editing the git Portfile.
#/opt/local/var/macports/sources/rsync.macports.org/release/ports/devel/git-core/Portfile
patchfiles patch-Makefile.diff patch-http.h.diff # Before
patchfiles patch-Makefile.diff # After
I'm not sure how to tell MacPorts to recompile git. I cheapened out and added a variant to force a recompile.
sudo port deactivate git-core @1.6.0_1+doc+svn
sudo port install git-core +doc +svn +bash_completion
Once that was resolved, I encountered yet another error.
$ git push upload master
error: Cannot access URL https://secure2.svnrepository.com/username/repository_name/, return code 1
error: failed to push some refs to 'https://secure2.svnrepository.com/username/repository_name/'
Turned out that curl needs to be compiled with the ssl variant. Normally, this should work:
$ sudo port install curl +ssl
---> Fetching curl-ca-bundle
---> Attempting to fetch certdata-1.48.txt from http://lxr.mozilla.org/seamonkey/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1
---> Verifying checksum(s) for curl-ca-bundle
Error: Checksum (md5) mismatch for certdata-1.48.txt
Error: Checksum (sha1) mismatch for certdata-1.48.txt
Error: Checksum (rmd160) mismatch for certdata-1.48.txt
Error: Target org.macports.checksum returned: Unable to verify file checksums
Error: The following dependencies failed to build: curl-ca-bundle
Error: Status 1 encountered during processing.
MacPorts should already have this fixed but just in case, I got around the error by skipping checksums.
sudo port install curl +ssl checksum.skip=yes
Deployment Issues
Here's the git related entries in my Capistrano deployment script:
set :repository, "https://secure2.svnrepository.com/g_username/project_name"
set :scm, 'git'
set :deploy_via, :remote_cache
set :branch, "master"
set :git_shallow_clone, 1
set :git_enable_submodules, 1
default_environment['GIT_SSL_NO_VERIFY'] = 'true'
But deployment failed midway:
$ cap staging deploy
Initialized empty Git repository in /home/username/project_name/shared/cached-copy/vendor/plugins/acts_as_list/.git/
github.com[0: 65.74.177.129]: errno=Connection timed out
** fatal: unable to connect a socket (Connection timed out)
** Clone of 'git://github.com/rails/acts_as_list.git' into submodule path 'vendor/plugins/acts_as_list' failed
The server firewall was getting in the way so I punched a hole for the git protocol:
iptables -I OUTPUT -p tcp --dport 9418 -j ACCEPT
Sweet Oblivion
With that all out of the way, I can now successfully deploy to the server the way I want to. Now I can get back to work :)
We are working feverishly our on new git only control panel which will use git over ssh instead of http/http. It will also have full trac integration, an s3 backup option, lighthouse and other hooks and a new look. The best part is, it will be included with your RailsPlayground service just like svnrepository or you can purchase access to it separately if you don't have a RailsPlayground account. Look for its release early next month.
That's awesome, Joe! It just keeps getting better and better. Looking forward to the new features.
Thanks 4 all :)