Git Submodules, External Repositories, and Deployment

Posted by Vincent Woo Tue, 26 Aug 2008 07:41:00 GMT

Even though I switched a few of my projects from Subversion to Git a couple of months ago, I still had lots to learn. So I dedicated last weekend to use submodules for Rails plugins. Unfortunately, it wasn’t the quick conversion I thought it would be but it turned out for the best.

Git Submodules

External repository references was handled so elegantly by Piston for Subversion repositories that I was willing to wait for its Git support. But I came across this blog post that dispelled misconceptions and it turns out submodules act just like Piston. In short, submodules will keep track of which revision it is on and won’t hit external resources unless it has to. Sweet! Submodules are great and simple to work with on the development side. Do:
  git submodule add git://github.com/rails/acts_as_list.git vendor/plugins/acts_as_list
  git submodule init
  git submodule update

And you’re golden. But there is still much to do. In order to deploy efficiently with Capistrano, I’d needed to do a “set :deploy_via, :remote_cache”. Onwards in search of a Internet accessible Git repository that I can call my own.

Git Repository Hosting

Enter Railsplayground.com Ruby on Rails Hosting. They have been hosting this blog for several years and have always been great for shared hosting. Bonus points for free Subversion and Trac hosting, up to 1GB disk space, with any hosting account.

Little did I know that they have been doing Git repository hosting for the past several months. Their homepage doesn’t mention it although their forum as well as their sister site do.

If you’re thinking about their git repository hosting solution, there are a couple of things to keep in mind.

PRO: Custom control panel: easy to create/delete git repositories and manage user accounts that has access to them

PRO: Trac integration is underway

CON: The only protocols they support are HTTP and HTTPS.

CON: It requires creating a ~/.netrc with my login and password. I don’t want to store any passwords in plain text on my laptop so I symlinked the file to an encrypted vault where my project files are anyways

MacPorts Pain

I’m using MacPorts on my Macbook Pro to compile git. This setup worked fine before but I couldn’t access the newly created git repository.
  $ git push upload master
  fatal: git-push is not available for http/https repository when not compiled with USE_CURL_MULTI
  error: failed to push some refs to 'https://secure2.svnrepository.com/g_vwoo/jamieliu/'
Unfortunately, MacPorts added in a patch to remove the USE_CURL_MULTI flag for stability reasons. Since the bug is triggered by a very large checkout on a powerpc mac, not applicable in my case, I removed the patch by editing the git Portfile.
  #/opt/local/var/macports/sources/rsync.macports.org/release/ports/devel/git-core/Portfile

  patchfiles   patch-Makefile.diff patch-http.h.diff  # Before
  patchfiles   patch-Makefile.diff  # After
I’m not sure how to tell MacPorts to recompile git. I cheapened out and added a variant to force a recompile.
  sudo port deactivate git-core @1.6.0_1+doc+svn
  sudo port install git-core +doc +svn +bash_completion
Once that was resolved, I encountered yet another error.
  $ git push upload master
  error: Cannot access URL https://secure2.svnrepository.com/username/repository_name/, return code 1
  error: failed to push some refs to 'https://secure2.svnrepository.com/username/repository_name/'
Turned out that curl needs to be compiled with the ssl variant. Normally, this should work:
  $ sudo port install curl +ssl

  --->  Fetching curl-ca-bundle
  --->  Attempting to fetch certdata-1.48.txt from http://lxr.mozilla.org/seamonkey/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1
  --->  Verifying checksum(s) for curl-ca-bundle
  Error: Checksum (md5) mismatch for certdata-1.48.txt
  Error: Checksum (sha1) mismatch for certdata-1.48.txt
  Error: Checksum (rmd160) mismatch for certdata-1.48.txt
  Error: Target org.macports.checksum returned: Unable to verify file checksums
  Error: The following dependencies failed to build: curl-ca-bundle
  Error: Status 1 encountered during processing.
MacPorts should already have this fixed but just in case, I got around the error by skipping checksums.
  sudo port install curl +ssl checksum.skip=yes

Deployment Issues

Here’s the git related entries in my Capistrano deployment script:
  set :repository, "https://secure2.svnrepository.com/g_username/project_name"
  set :scm, 'git'
  set :deploy_via, :remote_cache
  set :branch, "master"
  set :git_shallow_clone, 1
  set :git_enable_submodules, 1
  default_environment['GIT_SSL_NO_VERIFY'] = 'true'
But deployment failed midway:
  $ cap staging deploy
  Initialized empty Git repository in /home/username/project_name/shared/cached-copy/vendor/plugins/acts_as_list/.git/
  github.com[0: 65.74.177.129]: errno=Connection timed out
  ** fatal: unable to connect a socket (Connection timed out)
  ** Clone of 'git://github.com/rails/acts_as_list.git' into submodule path 'vendor/plugins/acts_as_list' failed
The server firewall was getting in the way so I punched a hole for the git protocol:
  iptables -I OUTPUT -p tcp --dport 9418 -j ACCEPT

Sweet Oblivion

With that all out of the way, I can now successfully deploy to the server the way I want to. Now I can get back to work :)

Posted in ,  | Tags , , , , , , , , , , ,  | 2 comments | no trackbacks

RSpec Tips

Posted by Vincent Woo Tue, 15 Apr 2008 21:53:00 GMT

Since I’ve been following Jamis Buck’s advice to limit the brittleness of view tests, development has been much more productive. Test the things that matter most else you’ll end up with diminishing returns and the week passed by without any productive work done.

I’m taking one step further and I’ve decided not to write RSpec view specifications.

I will still continue to occasionally test important view elements. It just isn’t worth the extra baggage of writing and maintaining another set of specs for every single view. I’m feeling the pain there and I’m taking steps to eliminating it.

But it is also painful to sprinkle “integrate_views” syntactic vinegar throughout controller specifications. I don’t want to do without that since I would still get notified of template errors. Unfortunately, this is not the recommended behavior and not much was written on the subject.

Peter Marklund has a solution but it didn’t work for me. Maybe because RSpec was updated since his blog post was written but it is more likely that I’m missing something obvious. However, I dug around the plugin and was happy to discover a simple method override that worked. Add to spec_helper.rb:

module Spec
  module Rails
    module Example
      class ControllerExampleGroup
        module ControllerInstanceMethods
          def integrate_views?
            true
          end
        end
      end
    end
  end
end

btw, while browsing the RSpec mailing list, I came across a post that explains a way to slightly speed up test runs.

Posted in  | Tags , , ,  | no comments | no trackbacks

Rake task runs both Test::Unit and RSpec tests

Posted by Vincent Woo Sat, 19 Jan 2008 03:04:00 GMT

I’m in the middle of converting tests from Test::Unit to RSpec in an older Rails project. So far so good although I’m sure that would change as I dive into more complicated test cases. Still, I’m hoping the pain is worth it for the multiple contexts and lessened fixture reliance.

A nifty feature that I accidentally stumbled upon is that the default rake task would run the test in both test and spec directories. It’s not like I have to exclusively choose one or the other.

That opens up lots of test conversion options. I could convert the tests all at once or a little at a time. Or I could forget about conversion completely and merely write new tests in RSpec.

It’s little surprises like this that brightens up my workday.

UPDATE (FEB8,08): RSpec tests won’t run unless all Test::Unit tests passes.

Posted in  | Tags , , , ,  | no comments | no trackbacks

Early Christmas Gift: Rails 2.0 Release Imminent

Posted by Vincent Woo Thu, 06 Dec 2007 21:53:00 GMT

Rejoice, for Rails Changeset 8321 indicates today as the day Rails 2.0 will be released.

Posted in  | no comments | no trackbacks

Rails String Core Extensions Cheatsheet

Posted by Vincent Woo Mon, 16 Jul 2007 22:43:00 GMT

PNG PDF



References
/CoreExtensions/String/Access.html
/CoreExtensions/String/Conversions.html
/CoreExtensions/String/Inflections.html
/CoreExtensions/String/Iterators.html
/CoreExtensions/String/StartsEndsWith.html
/CoreExtensions/String/Unicode.html
/CoreExtensions/Time/Conversions.html
Changing default date format in Rails @ Ruby Forum
Rails Rubyisms Advent @ errtheblog.com

Posted in ,  | Tags , , , , ,  | 1 comment | no trackbacks

Revisory Project Reflection

Posted by Vincent Woo Sun, 08 Jul 2007 23:59:00 GMT

Demo movie [1MB]. Revisory: Summer 2006.

Description

Online discussion space on graphical assets or unpolished concepts through message threads and image manipulation tools. Revisory facilitates non-linear design processes between people in distributed teams to better explore brainstormed ideas.

Development

The primary goal of this project to begin an introductory exploration of various web scripting technologies:
  • Ruby on Rails
  • HTML5 canvas element
  • Prototype and script.aculo.us frameworks
  • Javascript

The initial technical feasibility assessment concluded that the project was not commercially viable. A crucial factor was that the canvas element lacked current and future support by Internet Explorer. However, it was included in the latest version of Firefox, Safari, and Opera at the time. A decision was made to build a simple conceptional tech prototype as a learning experience through a project lifecycle.

Athestics is based on the simplicity of a blank canvas. The low key design encourages quick freeform sketches regardless of artistic ability. With its monochrome nature, the site design should fall back into the background and stand in contrast to the color possibilities of the user generated content.

End Results

Some testing was conducted near the end of this experiment. Revisory was utilized in highlighting the various design considerations given to a dynamic PDF document for review by a client. We found that keeping images updated and in context as per discussion point through appropriate zooming, panning, and highlighting held great value. The web app was relevant when visual communication was desired.

One area needing of improvement is in the generalized drawing tools. Most marks were kept simple to merely augment the existing image. Also, tools do not suggest efficient uses. Instead, future iterations should involve testing for more common behaviors to automate before any additional development starts.


Image used in demo is “More Fun Than Tekken” by Sevenflow and Kultdesign.

Posted in , ,  | Tags , ,  | no comments | no trackbacks

Automating Credentials in Capistrano

Posted by Vincent Woo Wed, 13 Jun 2007 20:40:00 GMT

Here’s a small tip that’s working well for me. If the user and password variables are set in the capistrano deployment scripts, then capistrano won’t prompt for the credentials anymore. It’s best to store them in ~/.caprc so you don’t commit your credentials to your SCM for all to see.

I’ve accumulated several different sets of credentials now that I’ve been working on several rails projects with different teams with their own login naming conventions. The simple assignment in .caprc just won’t do anymore.

Well, .caprc is just ruby code. Additionally, proc’s can be assigned to values to defer code execution until the value is first accessed. That way, the proc will have access to the application value in it’s local scope. Then we could do something like the following in .caprc:

set :user, Proc.new {
                case application
                   when 'project1': 'user1'
                   when 'project2': 'user2'
                 end
}

set :password, Proc.new {
                 case application
                   when 'project1': 'pass1'
                   when 'project2': 'pass2'
                 end
}

set :svn_username, Proc.new {
                 case application
                   when 'project1': 'svn_user1'
                   when 'project2': 'svn_user2'
                 end
}

Posted in  | Tags , , ,  | no comments | no trackbacks

Active Resource Cheatsheet

Posted by Vincent Woo Thu, 17 May 2007 03:50:00 GMT


PNG | PDF


References
The Ins and Outs of ActiveResource
ActiveResource Gets Custom Methods
Active Resource Tutorial
Taking ARes Out For a Test Drive
active resource rdoc

Posted in ,  | Tags , , , , ,  | no comments | no trackbacks

Goodbye rhtml

Posted by Vincent Woo Sun, 29 Apr 2007 04:57:00 GMT

What’s the one thing that’s more sacred to a man than anything else in the world? HAML. I’ve converted one template over and I’m sold. Just a few simple rules was enough to get started and half of them are based on standard CSS selectors.

With HAML, < and >’s are refreshingly gone but at the same time I felt like I lost a thumb. Angle bracket clusters, as cluttered as they are, were useful for highlighting the various elements on the page for easy scanning. Using a syntax highlighting plugin should help ease the transition.

Regardless, it just looks so clean. I’m looking forward to working more with HAML.

Posted in  | Tags , , ,  | no comments | no trackbacks

Validating Date and Time Objects in Rails

Posted by Vincent Woo Sat, 23 Dec 2006 04:07:00 GMT

How does Rails handle non-existing date input? If it’s obviously wrong, such as Dec 345, it’ll error out. If fed Feb 31, however, it’ll try to adjust and goes along its merry way saving Mar 3 to the database without any notice at all. I find that unacceptable and that problem would go away if we could coerce the model into keeping all the individual date param pieces.

We could hijack active record’s multiparameter processing into saving the result into an instance variable:
 def extract_callstack_for_multiparameter_attributes(pairs)
   @multiparameter_attributes = super
 end

And then use them in our custom validations:

if @multiparameter_attributes
  args = @multiparameter_attributes['birthdate'] 
  if args && Date.valid_civil?(*args).nil?
    errors.add('birthdate', args.join('-') + " is not a valid civil date")
  end
end

Posted in  | Tags , ,  | no comments | no trackbacks

Older posts: 1 2