Harvesting Codeplex Statistics
Update: The method described below does not work anymore. The page on CodePlex that was POSTed to was closed to external queries and so doesn't allow CORS any longer... Alas, it was fun while it lasted!
For a while now, all MOBZystems code is hosted on Codeplex. That has really proven to be a good move, if only for the download statistics Codeplex provides for each project. There is a Statistics page containing detailed historic information about the project, including graphs. That's fascinating information.
Harvesting Codeplex statistics for a single project
But of course, I need (want?) more: an aggregated view of all our projects on Codeplex. But there's no way to get that from Codeplex - that is: not from the web site itself. The number of page views, visits, and downloads is shown on every project page, though, for the last 7 or 30 days, or the lifetime of the project. That information is harvested using the following Javascript code:
$(document).ready(function () {
$('#PageActivity .Link').click(function () {
var val = $(this).attr('d:value');
$.post(
'http://runnet.codeplex.com/stats/getActivity',
{ period: val },
function (data) {
$('#PageActivity span').hide();
$('#PageActivity .Link').show();
$('#Page' + val + 'Label').show();
$('#Page' + val + 'Link').hide();
$('#pageViews').text(data.PageViews);
$('#visits').text(data.Visits);
$('#downloads').text(data.Downloads);
},
'json'
);
return false;
});
});
Basically, that means we can harvest those statistics as well, by POSTing to http://runnet.codeplex.com/stats/getActivity and supplying a period argument. Some more digging around reveals that the argument can be 7, 30 or -1 for 'All'.
Aggregating Codeplex statistics
The following code snippet is the heart of a console application that retrieves those statistics for a series of projects (the ones we host on Codeplex, of course):
''' <summary>
''' Download and show Codeplex statistics on a series of projects
''' </summary>
Sub Main()
' Show header
Console.WriteLine(
String.Format(
"{0,-40} {1,15} {2,15} {3,15}",
"Project:",
"Page views:",
"Visits:",
"Downloads:"
)
)
' Loop over projects
For Each projectName As String In
{
"MOBZHash",
"MOBZHunt",
"MOBZKeys",
"MOBZoom",
"MOBZPing",
"MOBZRuler",
"MOBZync",
"QuickCode",
"RegFind",
"RegName",
"RunNET",
"SeeThroughWindows",
"ShellRunner"
}
' Get activity information for a project
' Note: -1 is 'All'
Dim actInfo As ActivityInfo = GetActivityForProject(projectName, -1)
' Show activity info
Console.WriteLine(
String.Format(
"{0,-40} {1,15} {2,15} {3,15}",
projectName,
actInfo.pageViews,
actInfo.visits,
actInfo.downloads
)
)
Next
End Sub
The ActivityInfo class is simply defined as:
''' <summary>
''' Data returned from GetActivity
''' </summary>
Private Class ActivityInfo
Public pageViews As Integer
Public visits As Integer
Public downloads As Integer
End Class
The code that does the heavy lifting is basically adapted from MSDN. It creates a POST web request to /stats/getActivity on the project's domain on Codeplex, adds a period argument to the POST data and reads the response into a string. That string is then deserialized into an ActivityInfo object using JSON.NET.
Here's the code:
''' <summary>
''' Get the activity information for a Codeplex project
''' </summary>
''' <param name="projectName">The name of the project (as in [projectname].codeplex.com)</param>
''' <param name="period">7, 30 or -1 ("All")</param>
''' <returns>A populated ActivityInfo object</returns>
''' <remarks>No error handling!</remarks>
Private Function GetActivityForProject(
projectName As String,
period As Integer
) As ActivityInfo
' Set up a WebRequest
Dim request As WebRequest = WebRequest.Create(
String.Format("http://{0}.codeplex.com/stats/getActivity", projectName)
)
' Use POST to supply the parameters
request.Method = "POST"
request.ContentType = "application/x-www-form-urlencoded"
' Set up the POST data
Dim postData As String = String.Format("period={0}", period)
' Write it to the request stream in UTF8 format
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
' Set Content length BEFORE writing
request.ContentLength = byteArray.Length
Using dataStream As Stream = request.GetRequestStream()
dataStream.Write(byteArray, 0, byteArray.Length)
End Using
' Get and process the response
Using response As WebResponse = request.GetResponse()
Using dataStream As Stream = response.GetResponseStream()
Using reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()
' Return an ActivityInfo from the content of the response
Return JsonConvert.DeserializeObject(Of ActivityInfo)(
responseFromServer
)
End Using
End Using
End Using
End Function
The output of the resulting console application (the complete VB.NET code can be found in Codeplex Stats) is:
Project: Page views: Visits: Downloads:
MOBZHash 25 6 3
MOBZHunt 2040 1197 1207
MOBZKeys 28 7 3
MOBZoom 428 210 267
MOBZPing 672 248 256
MOBZRuler 776 429 540
MOBZync 3558 2325 2808
QuickCode 477 175 145
RegFind 844 322 249
RegName 1036 569 731
RunNET 162 79 40
SeeThroughWindows 9243 6127 6336
ShellRunner 535 144 182
Of course, the data presented here must still be imported into a database or a spreadsheet to be useful over time, but that's left as an exercise for the reader ;-)
As usual: enjoy!
PS: Since there is - to my knowledge - no public API to get these statistics, the usual caveat applies: the interface with /stats/getActivity may change without notice, so any code that depends on it (i.e. all of the above) may break anytime.