PERFORCE/CVS benchmark - not extremely scientific.

An experiment.

	I just took the time to try out CVS and PERFORCE over the following
	links:
		28.8 kbit/sec modem, 
		64 kbit/sec ISDN,
		10 megabit/sec ethernet 
		locally

	The 28.8 link has ping times of 260ms, markedly slower than most
	56kb sites on the Internet.

	All but the 28.8 test was between FreeBSD machines.  The client
	in the 28.8 test was a Linux box.

	The data is 126 files in 41 directories totalling about 1MB (the
	Berkeley dbopen source).

Raw data (min:sec):

                         28.8Kb    64Kb      10Mb      local	 Notes
                         ------    ------    ------    ------    ------

first co        CVS      3:34      1:34      1:01       :50  	 1,2
of all files    PERFORCE 3:23      1:13       :10       :15    

no-op update    CVS       :39       :31       :35       :13  	 3
when up-to-date PERFORCE  :02       :02       :02       :02  

submit 1 file   CVS       :08       :04       :03       :02      
from leaf dir   PERFORCE  :06       :02       :01       :01      

submit 1 file   CVS       :40       :28       :40       :20 	 4
from root dir   PERFORCE  :06       :02       :01       :01  

submit 8 files  CVS       :39       :13       :12       :10      5
from leaf dir   PERFORCE  :31       :08       :04       :04      

label all files	CVS				        :48	 6
		PERFORCE			        :00.6


Summary:
    Over fast links, PERFORCE is 3-6 times faster when dealing with the whole tree.
    Over most links, PERFORCE is 3 times faster when dealing with a single file.
    On slow links, PERFORCE and CVS are the same at bulk file movement.

Analysis:

   (1)  In the bulk transfers, CVS seems to have some fixed cost
	that becomes more apparent over fast connections.  Over fast
	connections the gap between PERFORCE and CVS is remarkable, but at
	lower speed (28.8), the gap more or less closes.  Both PERFORCE and CVS
	have streaming protocols, so they aren't affected by the latency
	of the modem line.

	PERFORCE's checkout is much faster than CVS for three reasons:  First,
	it doesn't need to scour the RCS tree to find the head revisions
	of the files - it has that in a database.  Second, once PERFORCE knows
	what files to update it can stream data solidly: it doesn't take
	a breath at the beginning of each directory.  Third, while CVS
	apparently can deliver the head rev without invoking the RCS
	'co' program, PERFORCE has a complete RCS implementation linked in,
	and can produce any revision directly.  It uses a combination
	of mmap() and memchr( ptr, '@', len) to make quick work of
	parsing even large RCS files.  (Sorry if I am drooling here.)

   (2)  I can't exactly explain why the CVS initial checkout is
	faster locally than over the 10Mb ethernet, except to note that
	CVS seems to stop and smell the roses at the beginning of each
	directory, and there must be more roses on the ethernet.

	PERFORCE is slower locally than over the ethernet because of disk
	thrashing:  while it is updating the client workspace, it is
	also modifying server data to record the clients updated state.
	This wouldn't happen if I had more than one disk.

   (3)  CVS seems to have different modes for local vs.  remote
	operations.  In the local mode, it scours the repository and
	the working directories at the same time.  In the remote case,
	it seems to do the remote repository first and then use the
	result to drive its subsequent probe of the working directory.
	This scouring affects both global and local updates and commits.

	The transfer of data affects remote updates and commits.  For
	some reason, it is a dramatic difference even over the ethernet,
	though not much more over the slow net.

	PERFORCE's "update" command is substantially faster than CVS because
	the server has all relevant information in a database, and
	doesn't need to scour the disk or transfer any information.

   (4)  PERFORCE's "submit" command is faster than CVS for the same reason:
	the server has all relevant information and doesn't need to
	probe any disks.

   (5)  PERFORCE's "submit" may gain more ground on CVS because of its
	built-in RCS implementation: PERFORCE doesn't need to fork/exec a 
	RCS ci(1) for each file.

   (6)  PERFORCE's file labelling is orders of magnitude faster than CVS 
	because PERFORCE stores the label information in a database, while
	CVS puts it into each RCS file.