Bug Tracker Blog by Corey Trager

BugTracker.NET integration with Subversion

by Corey Trager 1. November 2007 01:41


I'm in the middle of enhancing BugTracker.NET's integration with Subversion. Several months ago I added some integration, in the form of a "post-commit" hook script that inserts rows into BugTracker.NET's DB when somebody does a checkin into Subversion and includes the bug id in the comment.    Then within BugTracker.NET, there was a page to display the added rows.   And that was it. 

I realized only this week that I could carry the integration much farther.  I'm still wondering why it took me so long to realize that.  Mental block.  Anyway, since the path and the revision are in the database, I can use those to go back to Subversion to gather more info, well after the post-commit moment, when somebody is interacting with the page.  Here's the enhanced page that shows the file revisions, now showing new links for "diff" and "hist".

 

 

 

 

Clicking on "diff" causes the web page to go back to the server and execute C# code which in turn executes "svn diff", captures the output, and turns it into HTML.    The result looks like this:

 

 

 

 

Clicking on "hist" is similar, but it runs the "svn log" command.  The result looks like this:

 

 


From this last page you can see the entire history of a file, view the source code, use the snv "blame" command to see who changed what line when, and select any two revisions to diff.

So,  here are some of the design decisions:

* How do I actually talk to Subversion? The first example code I looked at was WebSvn. It uses "svnlook" which requires local, direct access to the Subversion files.  To use it, BugTracker.NET would have to be installed on the same server as the Subversion repository.  Instead I am using the client-side "svn".  You still have to install the Subversion software on the IIS web server, but you don't need to have your Subversion repositories on it.

* How does a BugTracker.NET user configure the repository? Should it be in Web.config, so that one instance of BugTracker.NET is associated with exactly one Subversion repository? Or should it be at the project level, to allow for a model where each project might have a different repository?  Or a combo, a global setting that is the default, but overridable at the project level?   Right now, it's just at the Web.config level.  I'll take a YAGNI, wait-and-see approach, see what my users  ask for.

* What if anonymous read access is not specified for the repository? At what level should the user/password credentials be specified?  Global, in Web.config?  at the project level?  at the user level?  Same as above, right now it's just at the Web.config level.

* Does it make sense to add more pages so that BugTracker.NET can serve as a general Subversion client?  Able to traverse the folder tree? Not yet...

The trickiest part of the work - and the most visually rewarding -  was producing the visual side-by-side diff.   I issue a "diff" to get the Unified Diff Format info, and then two "cat" commands to get the two source files.  Then, using the Unified Diff Format as a driver, I iterate through the lines of each file, marking them up. It was tricky because I am inept.  I kept incrementing my indexes when I shouldn't have or vice versa.  

Below is an example of Unified Diff Format.  The "@@ -1,16 +1,17 @@" means, that what follows refers to lines 1-16 of the old revision (13) and lines 1-17 of the new revision. Each subsequent line starts with either a + for added to the new revision, - for deleted from the new revision, or space for unchanged.  A - followed immediately by a + can be thought of as a change. 

Index: a.txt
===================================================================
--- a.txt	(revision 13)
+++ a.txt	(revision 14)
@@ -1,16 +1,17 @@
+A NEW FIRST LINE
111
222
333
+ADDED AFTER 3
+ADDED AFTER 3 B
444
555
-666
777
888
-999
-AAA
-BBB
+BBB CHANGED
CCC
DDD
+ADDED AFTER D
EEE
FFF
ZZZZZZZZZZZZZ
@@ -38,18 +39,12 @@
ZZZZZZZZZZZZZ
ZZZZZZZZZZZZZ
ZZZZZZZZZZZZZ
+ZZZZZZZZZZZZZ
111
222
333
+ADDED AFTER 3
+ADDED AFTER 3 B
444
-555
-666
-777
-888
-999
-AAA
-BBB
-CCC
-DDD
EEE
-FFF
+FFF
So, that's where the Subversion integration is so far.  
 

Tags:

Comments

11/8/2007 2:20:35 AM #

This is nice, but it's reproducing the functionality of repository browsers.  Why not have the revision number be a link that forwards to a repository browsing page, instead of cluttering the bug tracker with detail before the user has asked for them?

Karl Fogel |

11/8/2007 3:04:15 AM #

Karl - What you just described will be an option in the next release of BugTracker.NET, which I'll post tonight, or tomorrow.   You can have the links to my pages, or the links to WebSvn (or some other web-based thing, but I only tested with WebSvn), or both.

As to why I bothered with mine, folks might not want to run a LAMP or WAMP stack on the same machine as the repository.   Correct me if I'm wrong, but WebSvn, because it uses svnlook, needs local file access to the repository, right?

Here's what I went through getting WebSvn up and running so that I could test my BugTracker.NET changes.  It wasn't bad, but not smooth either.   It was enough of a hassle that maybe folks will appreciate the integration built into BugTracker.NET:

* I had to re-educate myself about WAMPS (Windows, Apache, mySQL, php packages) and install one.  I installed XAMPP.  For whatever reason, "xampp-control.exe" doesn't launch right on my machine.  I can see it in taskmanager, but there's no window displayed.  Some sort of bug.   I looked in the xampp folder and saw "xampp_start.exe".  I tried it, and it worked.   Sort of.

* Well, yes started, but because I was running IIS on port 80, and because apache was configured to start on port 80, apache failed to start.  So, I had to hunt around for where to change the default port of 80.  That alone didn't work, because there was still a conflict over 443.   After configuring the ports, I was able to get apache to launch and I could browser to the xampp home page.

* Next I had to tell apache about WebSvn.  That is, do the apache equivalent of creating an IIS "virtual directory".  Even though I pretty much use just Microsoft tool at my real job, I've used apache before, so I knew I was looking for a file like httpd.conf, and that I would copy the boilerplate text in it.  I would just look for the lines for, say, phpMySql, and copy/alter them for WebSvn.   But the boiler plate I was looking for wasn't there.  I had to hunt.  Turned out to be in httpd-xampp.conf.

* Launched WebSvn and got an error "Cannot redeclare class Config in pear/config.php".  Sigh.  This took time.  I sppent a half hour reading discussion threads where others had the same problem.  Ended up renaming the pear/config.php based on what one guy did and that fixed it.  But, Gee, I wonder what I broke by doing that?

Then I went through the process of looking at the WebSvn specific config, which I didn't complete.  WebSvn relies on the *nix utilities diff and sed.  The WebSvn documentation recommends installing Cygwin.  I didn't bother, because I didn't need that to test my BugTracker.NET changes.



Corey Trager |

11/9/2007 11:42:59 AM #

nice job!

DamPee |

11/9/2007 7:15:01 PM #

So far I love what you have done the integration of Subversion into the project was a huge plus while I was evaluating bug tracking systems. The only problem I have so far and it is kind of a minor one I guess, is having to make sure that the other developers working on a project format their commit comments properly. I have seen other systems which utilize the following in TortoiseSVN and I am really curious as to if this could be a possible future feature.

tortoisesvn.net/.../tsvn-dug-bugtracker.html

A setup like this within TortoiseSVN would be great. It would eleviate me having to mandate down certain ways of formating comments as well as give a direct link to the issue's webpage from TortoiseSVN.

Not sure how this would work with the system you have built so far just bringing it up mainly as a feature request for the Subversion integration your working on.

Joe Prochazka |

11/9/2007 7:17:02 PM #

Sorry to post twice in a row but I think i pasted in the wrong link to my forst comment so here it is.
http://tortoisesvn.net/issuetracker_integration

Joe Prochazka |

11/9/2007 8:33:59 PM #

Some more documentation, courtesy of my "competitor":
citydesknews.com/.../TortoiseSVN.html

The following worked for me:

Copy these lines into a .bat file.  Edit the "YOUR BUGTRACKER.NET URL" part, and run it.  You can change the label and message to what you want.   The "bug id: " and the number will get appended to the end of the checkin message.  

svn propset -R bugtraq:label "BugTracker.NET ID:" .
svn propset -R bugtraq:url "http://YOUR BUGTRACKER.NET URL/edit_bug.aspx?id=%BUGID%" .
svn propset -R bugtraq:message "bug id: %BUGID%" .
svn propset -R bugtraq:number "true" .
svn propset -R bugtraq:warnifnoissue "true" .
svn commit -q -m "Added BugTracker.NET properties"

Then, in the post-commit vbs script, replace the lines that look for the digits at the start of the first line with these that look for the digits at the end of the last line.

'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' Look for the bugid as the first digits in the checkin comment
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
dim re
set re = new regexp
re.pattern = "[0-9]*$"

dim matches
set matches = re.execute(lines(ubound(lines)-2))




Corey Trager |

11/9/2007 8:44:02 PM #

(Thanks for the explanation; it makes sense.)

-Karl

Karl Fogel |

11/9/2007 8:56:27 PM #

**** IMPORTANT! ***   In my instructions above, do *NOT* copy to a .bat file.   Instead, type in each line from the command line.

Corey Trager |

11/9/2007 9:00:18 PM #

***** Oh, you can use a .bat file, but change the %BUGID% to %%BUGID%% in both places.

Corey Trager |

11/9/2007 10:43:20 PM #

Very nice thanks alot!
Will give it a shot thanks a bunch.

Joe Prochazka |

11/15/2007 4:57:43 PM #

Hi

Just a quick vote for respositorys being specified at project level. In our case we have one instance of bugtracker covering multiple projects and each project has it's own repository.

Jon Hilton |

11/16/2007 11:10:15 PM #

Came back to say thanks for integrating the "Bug ID:" field solution into you latest version works great.

Also I'll cast a vote as well for Jon's post. At the moment this is the only thing holding me up from placing the system into production. The only issue with multiple repositories seems to be with the view diff and review histroy links looking for '/svn/trunk/Default.htm' instead of '/svn/ProjectName/trunk/Default.htm' when using the svn command.

Joe Prochazka |

11/18/2007 2:28:59 AM #

Jon, Joe - Just posted version 2.6.4 at Sourceforge.  It includes the ability to specify your repository settings at the PROJECT level.

Corey Trager |

11/19/2007 4:39:30 PM #

What more can I say thanks again for your time and effort!

Joe Prochazka |

11/19/2007 7:07:32 PM #

Yep really loving the project-level integration.

However I'm having trouble with the post-commit hook script.

I can't pin it down but basically I'm not getting commit records added to the database.

Our Bugtracker server has a mapped drive link to the subversion repository (Q:\SVN)

Is the post-commit script itself executed on the subversion server. The reason I ask is because a command prompt window flashes up on my client machine when I do a commit but I was under the impression svnlook had to be executed on the subversion server itself?

Jon Hilton |

11/19/2007 7:18:04 PM #

Jon - Yes, the post-commit script itself is executed by the subversion server process.  That's odd about a command prompt window flashing on your client machine when you do a commit.  What subversion client are you using?

If you crack open my post-commit script, you can change the "debug = false" to "debug = true" and then run it directly from the command line and figure out what's happening.

Is your script in sync with where the bugid is appearning in the comments?  My code either looks for it at the beginning or at the end - see the "re.pattern" related code.

Corey Trager |

11/20/2007 10:44:55 AM #

I'm using Tortoise SVN.

I will try your suggestion re debugging.

Jon Hilton |

11/20/2007 7:10:59 PM #

Further investigation has revealed that accessed file-base repositories on mapped drives causes some strange effects. As far as I can tell the post-commit bat file tries to run locally on my machine because my checked-out copy of the repository is pointing to a mapped drive using the file:/// prefix.

Having installed svnserve I am having considerably more success accessing the repository using svn://

Jon Hilton |

1/14/2008 10:36:52 AM #

Can you tell me how to set the post-commit if I use the http protocol for subversion? thanks!

Calvin |

1/14/2008 3:40:38 PM #

The vbs script must be on the same machine as the repository, otherwise it won't get fired.
Also, it executes "svnlook" which uses local file access.

See:
http://svnbook.red-bean.com/en/1.1/ch09s03.html

Corey Trager |

7/10/2008 4:36:57 PM #

Hi,

I'm testing the subversion integration, with the properties and post commit script.
TortoiseSVN don't finish the commit, but update the content. Something wrong? Some ideas?
Info:
- Server: Windows Server 2003 EE SP1
-- VisualSVN Server 1.5.1
- Client: Windows XP Professional SP3
- TortoiseSVN 1.5.0

Best regards...

Luiz Augusto Rodrigues
Web Developer - ASP.NET 2.0 C#
UGIST-SGI/MS
Brazil

Luiz Augusto Rodrigues |

7/25/2008 10:01:39 PM #

Hi,

I have started using BugTracker and integrated it with SVN. It works very well and I am very pleased with it, but I found something weird I can't understand why it is happening.

Basically I found that one bug had associated revisions even though nothing was checked-in with that bug-id. And when I clicked to see what files were checked-in, I found there were files for a completely different project (and they are not using bugtracker at all).

I went to the database to see what was going on, and I found a lot of entries in the revisions table, with their bugids set to a "random" number and no ids in their comments. One of these entries happened to have the same bugid of the bug in question.

I can't figure out how this could have happened, can you assist me with this?

thanks

Iang Yim |

7/28/2008 2:03:49 PM #

Nobody else has reported a problem to me like the one you are describing.   I think you need to look at the hook script, read the code, and turn on the debugging code, or add more debugging code.   I think it's going to turn out to be something unusual in your configuration that I didn't anticipate or a configuration mistake by you.

Corey Trager |

8/5/2008 10:18:55 PM #

Hi,   
I'd like to inform that could hold the settings and i'm using the SubVersion integration.
The problem was the script of configuration and permissions of the repository.
Details:
- VisualSVN Server 1.5.1
Using Windows authentication, add local user BUG and read only permission on the repository.

- TortoiseSVN 1.5.0
Batch file for integration not works, because svn.exe not exists, Tortoise 1.5.0 encapsulated in DDL file. So using the properties in folder options, add the configuration (propset) and execute commit manually.

Now works fine!!!
I will accompany the site for new versions and corrections, especially regarding MS Active Directory Authentication.

Best Regards,

Luiz Augusto Rodrigues
Web Developer - ASP.NET 2.0 C#
UGIST-SGI/MS
Brazil

Luiz Augusto Rodrigues |

9/20/2008 1:00:18 AM #

Does bugtracker integrate with VisualSVN?

Mike |

9/25/2008 3:37:51 AM #

Hey, this is a really nice tool. The SVN integration was what really interested me.

Speaking of the SVN integration, I seem to having problems linking the SVN revision numbers to ticket IDs. If I attempt to manually link a revision inside of BTNET to a specific bug, I receive the following error:

Error executing svn.exe command from web server.
svn: PROPFIND request failed on '/Repositories/GUI' svn: PROPFIND of '/Repositories/GUI': Server certificate verification failed: issuer is not trusted (https://localhost)
Arguments passed to svn.exe (except user/password):log https://localhost/Repositories/GUI/ -r 51 -v --non-interactive

We are using SSL along with SVN, so I guess I see understand the cause of the problem. The ASP.NET worker process on our web server doesn't trust our SVN SSL certificate. What I don't understand is how to fix it or work around it. I have unsuccessfully attempted to get the ASP.NET worker process on the BTNET server to trust our SSL certificate using the WinHttpCertCfg tool. This doesn't seem to solve the problem. It would be nice if I could influence BTNET to not use the "--non-interactive" switch in it's SVN commands, because I believe I would then be prompted to accept/reject the SSL certificate, and could then continue along.

I sure would appreciate any advice you can lend; the SVN integration is a really key component for us.

Thanks,
-Skel

Skel |

9/30/2008 1:46:00 PM #

Hi
It took me almost a day to make Bugtracker.net work with SVN
(not 100% though  diff and history are still not working when we browse the repository.)
Diff is a big problem, cannot handle long directory names.. problem is WebSVN uses diff.exe

My main problem is your post-commit.vbs hook.
I am not quite familair with windows permission setting.
When I commit from remote machine, the shell.exe(cmd) gives an error, permission denied.
Any help would be great.
Ajay

Ajay Wazir |

9/30/2008 2:21:26 PM #

If you need help with BugTracker.NET, post your question here please:
sourceforge.net/forum/forum.php?forum_id=226938

Corey Trager |

Comments are closed

Powered by BlogEngine.NET 1.5.0.7

RecentComments

Comment RSS