Free Videos about Mastering Views

Overview: Drupal CVS/SVN Setup

Tagged:  
Play Video
Movie link (right-click to download)
Problem with this video? Contact me

Setting up a Drupal deployment, from a "where do I put the code" standpoint, is different than the setup within your actual Drupal install. It's based, quite a bit, on what you know with regards to technologies and your comfort level with them.

For example, when I first started using computers, I was Ok with using FTP. Software allowed me to connect and upload with ease. I didn't have to know the command line and frankly, knowing commands in a terminal scared me.

Flash forward a number of years and now I can't live with out it. It's just so fast. Using the command line and technologies like CVS and SVN is what allows an admin to download, update and maintain an installed codebased with much ease - but, this comes at the price of having to learn those technologies.

So, here's your warning! This content is very much on the geeky side. It outlines using the CVS/SVN combination which many of the top Drupal developers use. In this video, I outline my repository structure and how I deploy my Drupal code. The first half provides an intro with links to stuff you need to know about CVS and SVN, and the later walks through some of the details of how it all works.

As mentioned in the video, I reference my other video about multisites and I also provide some pointers about knowing how to do things like SVN Filtering for adding and removing files which are added/removed from code repositories.

Also, here's a great link on Drupal.org with more detailed info about CVS with Drupal . Pay special attention to the -r switch for the command!

If you have suggestions about improving my system then I'd love to hear them! - Stay Drupal Addicted!

Very useful overall. I've been thinking a lot about how to use svn to setup some sites I'm working on and ended up with a similar system.

One caveat - towards the end of the video, you suggest using svn up to see what revision an external is at. This is quite dangerous, since it could update the checked out directory, which may not be your intent. Instead use the svn info command to do that. You could also use svn status -u to see what files, if any, would be modified by an executing svn update.

Thanks a lot for the pointer on svn info! Yes, you're right, it would be a bad idea to use svn up just to check what revision the repo is at. For anyone reading this, svn help is your friend (even mine) ;)

Excellent tutorial, thanks for sharing your knowledge.

Thanks for a great video! Very timely for me as we are researching different deployment workflows at our organization right now.

Hey Matt! Great article/video. I am currently trying to implement this right now but I'm a little fuzzy on some of the details
So far I have done a CVS checkout of core and all the modules I use and love. I understand creating the repos and linking them and all that jazz, but I'm unsure about the updating part.

If a new version of drupal comes out and I use CVS to update my core and modules, how do I then update my svn repos from there? When this is all complete, do I then do a fresh checkout on my production server and call it a day?

Can you show me your folder layout of your dev server and production server to show me where what goes where?

Thanks!

Hi GotDrupal.
Good Day,
I find your videos very good. It helps me alot in me learning drupal.
I just want to request if you could make a video about these.

http://gitcasts.com/posts/git-on-windows
http://mikkel.hoegh.org/blog/2008/a_git_mirror_for_drupal_cvs
http://www.versioncontrolblog.com/2007/08/02/upgrading-drupal-52-with-gi...
http://www.ruzee.com/blog/2008/10/drupal-development-and-deployment-usin...
http://drupal.org/node/349658
http://nathanj.github.com/gitguide/
http://sachachua.com/wp/2008/06/11/setting-up-your-drupal-development-en...

I'm now trying to learn msysgit and trying to link your video "Overview: Drupal CVS/SVN Setup" to it(msysgit).
I'm new to drupal and I want to learn the best way to set it up(server). I tried the basic way of copy and paste download and upload and now the pro way.

Thank you very much for your time.

One of your million fans,
Jefferson Long

Would it be better in the Site Root folder to add an svn:external to ../../sites and have sites checkedout without a symlink.? Or do a second manual checkout of sites within the drupal director? The reason I mention this is if you are working with a mix group of developers, the unix and mac guys are happy with the symlink but the windows guys are out of luck. Any thoughts on this? cheers

With svn:externals you have to remember to add --ignore-externals to svn update if you just want to update your core files.

The example I show in the video is one that fits my personal situation with regards to multi-site management. Hopefully, if you get the premise of using externals, you can bend them to your will. There are a few minor details like the other comment about ignoring (thanks @Oscar!), but it's all a matter of what your deployment, management and maintenance needs are.

Since most all of my sites stay on the same track with regards to modules used and deployment it works best this way for me - until I think of, or change to a more efficient method - Much Luck!

Hey,

Thanks for the video. I may have missed it in this video or something, but where are you using the CVS Deploy module? I've never used the module before but am curious to see it in use.

Great video.

You mention a tool for finder where you can right click a file to commit etc, what's the link to the tool?

Thanks for this video, it certainly seems like a very efficient setup and I must have watched the video about 10 times to try and really understand the concepts you are presenting here. Let me preface my questions by saying I am a SVN newbie so the questions may seem obvious to some.

1.) What about the trunk, branches and tags directories? Do all 4 different repos each have a trunk, branches and tags setup or is that structure not an essential part of the setup?

2.) It would be great to see an update to this video dealing with the newly launched SVN repository for Acquia and how something like that may fit into this setup. Do you have any advice for combining two different repos like Acquia for the drupal core files and basse modules and then a private repos for the sites folder?

Thanks for any assistance you may be able to offer
 Gregg

1.) What about the trunk, branches and tags directories? Do all 4 different repos each have a trunk, branches and tags setup or is that structure not an essential part of the setup?

One of the essential things to know about most all version control systems is that the use of Trunk, Branches and Tags is just convention (except for CVS - to my knowledge). CVS was one of the first version control systems and implemented this terminology. However, it is not a prerequisite with other systems like svn, git, bazzar. These later systems simply treat your version control like a bunch of folders, much like your desktop. You can use any naming/storage convention you want. Using Trunk/Branches/Tags is optional. Some developers will simply use HEAD and Release, while others will use the good old standard. The short answer to your question is "No".

2.) It would be great to see an update to this video dealing with the newly launched SVN repository for Acquia and how something like that may fit into this setup. Do you have any advice for combining two different repos like Acquia for the drupal core files and basse modules and then a private repos for the sites folder?

This is certainly possible. The big issue with using Acquia's repository is that you can't use SVN on the same code for two different repositories (that I know of). So you would need to use something like this.

  • Drupal.org (cvs) -> YOUR SIDE (svn | git | bzr)
  • Acquia (svn) -> YOUR SIDE (git | bzr | cvs)

Things can get pretty tricky when you've got a variety of systems in place. So blending the two might look like this.

Acquia (svn)---\
               |------YOUR SIDE (git)
Drupal (cvs)---/

Note that whatever is used on the left hand side, can't be used on your side (unless you write a script that will copy the svn repo to another folder and rip out the .svn folders)

Here's a find command to do that (within the same folder)

find . -name '.svn' -exec rm -rf {} \;

If you're new to using SVN then give it some time before things really start to sink in. The organization and management is something that will be refined over time.

Since I've shot this video my own has changed a bit. Here is how it looks now.

.svn
conf << this repo
contrib << external repo
custom << external repo
databases << this repo
drupal << external repo (using vendor branches)
helpers << this repo
offline << this repo
scripts << this repo
sites << external repo (symlinked within drupal)

As you can see, my one main repo, with a fully portable Drupal multi-site install simply uses externals pointing to four other repos on the same server (actually 5, one is embedded within sites).

  • conf contains dedicated apache.conf files about how the whole drupal setup is structured (this makes the whole repo portable to other servers)
  • contrib only contains modules directly pulled from cvs.drupal.org. No trunk, branches, tags. Modules are individually symlinked into sites within the sites folder. (symlinks are saved in svn - yeah!)
  • custom are my own modules where I use trunk/release naming - also symlinked (by release) into sites/sitename/modules
  • databases these are modified base install db's. (e.g. innodb version vs. myisam - these are not database backups)
  • drupal this is the web root folder (options for symlinks turned on)
  • helpers includes shared resources, like players for audio, video, wysiwyg editors, etc. (symlinked into modules like wysiwyg module)
  • offline includes the files to show when I take all sites offline for upgrading. See this video
  • scripts utility scripts used for management
  • sites contains yet another external repo to "default modules" which are included into /sites/all/modules. These are modules typically used on all sites. So, I essentially have two different modules repos. One is for commonly used and the other is for randomly used modules.

I hope this helps out and if you've got ideas, or other feedback about other ways to do things even better I'd love to have you comment here.

Matt

Hi Matt
I just wanted to post some interesting advice I received from the Acquia team with regards combining the Acquia SVN setup along with an externals based sites folder. I am in the process of trying this out but thought it may be useful to other visitors to see what they suggest. It seems to make sense to me.

http://acquia.com/node/877583

Thanks for your help and for making this video in the first place
 Gregg

Matt,

I have enjoyed watched your superb series of developer videos. The amount of time spent learning from scratch about CVS and SVN make me question how much time I am saving by not downloading/uploading via FTP, but I am too far into this now and refuse to throw away the time already invested. And with all I've read I am still missing the fundamentals.

You reference sources for svn installs and the svn book and I have read some chapters several times now and still have the same question. Where is all of this work taking place, on a local computer (like the one I am using to comment) or from a server?

In one of your videos you state that all this work can be done then sent to your server. That is where most of my time is being spent and I am unsure if it is necessary to work from my local machine. Besides it is driving me nuts trying to set up externals using TortoiseSVN (I am on a PC) from a local machine. It does not seem to work using a url on a local machine and relative urls to the local repository returns errors. Although it seems logical to work on a local machine if you are doing a lot of code editing.

Enough wambling. When using the command line and entering file:/// is access going to the local machine or the server. The book states this is direct repository access (on local disk). I assume that to mean the server local disk. If correct then I must need to edit the etc/hosts file for my local machine? With what?

Back to reading the book some more, again.

I found this video useful understanding the big picture of using CVS and SVN, but I still don't understand how to do this stuff particularly the the SVN stuff.

I bought the Mac Versions Application thinking that would help since it is suppose to be a GUI for this, but I still don't get it (and can't get that program to work either).

Also, I like your series of videos for creating modules and it would be really helpful if you could group your videos in series like you did those.

Thanks for all the time you put into these tutorials, they are a HUGE help!

All the Best,
 Steve

If using Drush, don't issue commands from site_root/sites/mysite.com. Drush will give "installation directory could not be found" errors, which makes sense since the sites folder has been pulled out of the drupal tree.

You have to go "through" the symlink in the drupal folder for drush to not complain (eg, from this location: site_root/drupal/sites/mysite.com).

It's the little things which escape you every once in a while. You are perfectly, exactly correct. The environment of the shell (and drush) does need to know the right path and not going through the symlinks is what causes it not to work.

Very nice. Now, I'll have to shoot a video about using Drush. :)

at about the 19:15 point in the video you mention quickly something about excluding or ignoring all images in the sites/site-name/files/. i was wondering what you meant about "another backup solution" for the /files/ directory?

looking further into the /files/ structure, i was not sure if ignoring all the images makes sense in all instances. for example, a store that's using drupal & ubercart, you'd want to keep the product images in revision control, to facilitate between development & live servers, as well as having a full backup within svn, right? or will having hundreds (if not thousands) of images in svn bog things down to a point of making svn unusable?

assuming that i'd like to keep the "original" files in svn, but ignore all the copies / resized versions that get created (any time you upload a file), i'm not sure which ones to actually ignore. starting with imagecache, i'm assuming because of it's nature, that you can ignore all images in that directory, but since the imagecache module needs the original file, i'd want to commit and version control all of the original images, right? would i svn-ignore the contents of imagefield_thumbs, and other imagefield_* directories?

thanks the great video!

What you put into the version control is certainly up to you. So if you wanted Ubercart images you could put those in.

The situation of ignoring the whole files is one where you have a lot of content being added by a lot of other users and this is typically better handled using something like an Amazon S3 backup or using rsync to another server.

When you want to ignore files based on pattern matches you just need to learn how to use the svn:ignore property.

http://svnbook.red-bean.com/en/1.0/svn-book.html#svn-ch-7-sect-2.3.3

I hope this answers your question.

I'm trying to include the files directory in my repo but am running into user permissions problems.

I have to chown the files directory to my web server's user (www-data) for various things such as ctools and css optimization to work, but now I'm getting svn errors ("cannot open file ... /files/.svn/lock Permisssion denied").

Everything else in my drupal install is owned by my user (ie, not www-data). That may be wrong. But even if I chown everything to www-data, I still need to check in and update working as my user...

I'm guessing there's a way to maintain file ownership and permissions in SVN, but I haven't found how yet.... Thought I'd ask here. Any advice?

thanks!