Cricsheet

Explanation for withholding of Afghanistani matches

Thu, 14 Nov 2024 16:30:00 +0000

Today I’ve updated the site to remove all matches featuring the Afghanistani men’s team, as well as matches from the Afghanistani Premier League. I’ve thought about this for a while, and have finally decided that I need to do it. I don’t feel comfortable continuing to publish matches involving, or played in, Afghanistan while the Afghan women’s players are simply being ignored by the ICC and most of the full members (Cricket Australia being the notable exception). I’ve felt this way for a while, but the recent publication of the Women’s Future Tours for the next 5 years without even a passing mention of Afghanistan, along with the cricket media (again with a few honourable exceptions) not even mentioning Afghanistan in their reporting of the FTP, has finally persuaded me to remove Afghanistan from the site. The ICC continuing to ignore that Afghanistan is in breach of the Full Member requirements by not having a women’s team is also an irritation, along with the empty platitudes they spout every time they’re forced to comment.

The result of my decision to withhold matches, as I write this, is that 329 matches that were previously available within the downloadable data are now gone. These matches will hopefully be restored at some point, but only when I feel comfortable doing so. I don’t expect my small decision to make a difference to the ICC, but I’ve decided that that doesn’t necessarily matter. I’ve made the decision to do what little I can do with this site to protest the situation. Some people may disagree with this decision. I don’t particularly care. There are bigger issues at stake, and if my doing this one small thing can cause people to seriously think about the issue then it is worth much more to me than providing any match data could be. I’ve taken longer than I should have to come to this point, and others have called for action sooner, and more eloquently than I, but I’ve finally got there.

Monthnotes: April 2023

Mon, 01 May 2023 21:10:00 +0000

The 4th monthnotes in a row (let’s not mention that it’s 1 day late) finds us with another 133 matches added to the site bringing us up to 14,431 matches in total. The increase in the number of matches added this month (from the 108 added in March) is mostly down to the start of the County Championship, Rachael Heyhoe Flint Trophy, and the ongoing Indian Premier League. Apart from the matches that were added there were no major changes in the project this month, with the only change being work to cope with the IPL allowing reviews for no-balls and wides.

See you at the end of May, (or possibly start of June)!

Monthnotes: March 2023

Fri, 31 Mar 2023 18:10:00 +0000

First January, then February, and now, in a surprising move, I complete the hat-trick with the 3rd “monthnotes” entry. Another relatively quiet month for the project as I’ve been trying to make sure I get enough rest now that I’ve started a new job, however this doesn’t mean that nothing has happened.

Another 108 matches were added to the site during March, bringing the total to 14,286. Most important in those 108 matches are the 22 matches of the inaugural Women’s Premier League, which I’m very happy to be able to release the data for. April should prove to be a busier month for updates, with the IPL starting again (as I write this), and the English domestic season also getting underway.

See you at the end of April (hopefully)!

Monthnotes: February 2023

Tue, 28 Feb 2023 21:30:00 +0000

Hello again 👋. Well, it looks like I’ve managed to at least make it to the second of my planned month notes for the project, huzzah! It has been a quieter month for the project, as I’ve had other things to deal with (such as finding a new job, pesky layoffs!), but I’ve still made progress on something I mentioned last month.

New Women’s Test Matches

I’ve successfully converted 5 scorebooks from (the fantastic) Women’s Cricket History into data files, meaning that I’ve increased by 50% (from 10 to 15) the number of Women’s Test matches available from the site. The matches added were:

Sadly, for a variety of reasons, I’ve not been able to generate the data for the other missing Women’s Test matches. Some matches still have discrepancies even after checking the scorebooks, some of the scorebooks are actually just scorecards, and some scorebooks are too faded to be able to read. I’ll contact the various boards to see can I resolve the discrepancies, or acquire better copies of the scorebooks, but for now we’ll just have to put up with the missing data. In the meantime I plan to gradually work through the available ODI scorebooks to see whether I generate data from them. I may also finally get around to writing up some of the details of how I go about converting the scorebooks to data and putting it on my own site.

Monthnotes: January 2023

Mon, 30 Jan 2023 20:30:00 +0000

Inspired by Simon Willison, and his use of weeknotes, I’ve decided to start providing a semi-regular update on what I’ve been doing with regards to Cricsheet. As I know I’m very unlikely to be consistent enough to do this every week I’m starting out slowly (and hopefully more realistically) and have decided to go with “monthnotes” instead. As we reach the end of the end of the first month of 2023 now seems as good a time as any to start. With that in mind, I’m going to provide an update on what I’m been working on since late last year. Let’s see how this goes.

Adding previously missing tests

I’ve succeeded in adding the data for two Test matches which I had previously listed as “missing”, the 5th Ashes Test in January 2003, and the 4th India vs Australia Test in January 2004. In the former I was finally able to resolve an issue regarding the runs conceded by Lee and Gillespie in the 1st innings, and in the latter I identified the issue with the total number of boundaries listed for Martyn and Lee in the 4th innings.

I’m cautiously optimistic that the same method I used to help resolve those discrepancies can be used some other missing matches of that period, however it’s a painstaking process so I’ll just fit it in when I have nothing else to do.

Adding the Syed Mushtaq Ali Trophy

I’ve added data for 648 matches from the Syed Mushtaq Ali Trophy seasons for 2016 to 2021 seasons. I was able to do this due to the fortuitous discovery that the BCCI website has ball-by-ball information hidden within the JSON data for some matches on the site. It’s not perfect, and I’ve had to work around some interesting idiosyncrasies, however it was doable and I’m happy to have the match data available.

You may have noticed that I didn’t mention the most recent (2022) season of the Trophy. Sadly the JSON data for every match I’ve checked from that season is incomplete. Only minimal data is provided for each delivery and I can’t fill in the details. I’ll check on a regular basis to see if the situation changes, but until it does I won’t be able to add the most recent iteration of the competition.

Register updates

In order to be able to add data from the BCCI website I unsurprisingly needed to be able map player ids from the site to existing Cricsheet Register ids. This has resulted in the addition of some new columns to the register data, along with 1,329 ids mapping to existing entries.

A new outcome method - VJD

Another result of the addition of the Syed Mushtaq Ali Trophy to the site is that the various data formats (for example, JSON) now support another outcome method (VJD) which indicates that the winner of a match was determined by the Jayadevan system, an alternative to the Duckworth-Lewis-Stern system. This new system has been used in a number of seasons of the competition, and may well be used in the future, and now there is support.

Twitter and Mastodon

The project has joined the exodus from Twitter and moved to Mastodon. The old account had 1,574 followers, and the new one has just 17, however hopefully that will grow, and the account can be useful.

The new profile is at https://nitech.online/@cricsheet, or you can follow @cricsheet@nitech.online in your preferred Mastodon client. If you’re not ready to join Mastodon yourself but know about RSS you can subscribe to updates at https://nitech.online/@cricsheet.rss. The profile is now at https://social.deeden.co.uk/@cricsheet, or you can follow @cricsheet@deeden.co.uk in your preferred Mastodon client.

Adding matches from scorebooks

Finally I’m working on a small project to allow me to generate ball-by-ball data from scorebooks. This will allow me to convert some of the scorecards from the wonderful Women’s Cricket History into actual data files. I’m still ironing out the kinks, but it’s looking hopeful.

Review corrections

Sat, 04 Dec 2021 00:00:00 +0000

Well this is awkward. While doing some other work on the data files I noticed that one of the review entries didn’t seem to make sense. This, as it does when something like this happens, led me to write some extra validation checks, and that found a number of discrepancies. I’ve corrected them, and the new validations will now be run for every new data file generated. The details of the corrected matches are listed later.

If you make use of the review data then you’ll probably want to update your copies of the data files. With that in mind, I’ve created a download of the JSON data for the corrected matches.

Moved to a different delivery

I’d associated some reviews to the incorrect delivery in an over. Generally this was down to my failing to take wides or no-balls into account. I’ve corrected the following entries.

Date	Teams	Event	Reason
2009-11-24	New Zealand vs Pakistan	1st Test	Moved from the 3rd to the 4th delivery of the 126th over in the 1st innings. Neglected to account for the wide on the 2nd delivery of the over.
2013-11-01	Pakistan vs South Africa	2nd ODI	Moved from the 2nd to the 3rd delivery of the 48th over in the Pakistan innings. Neglected to account for the wide on the 1st delivery of the over.
2014-12-10	Sri Lanka vs England	5th ODI	Moved from the 6th to the 7rd delivery of the 45th over in the Sri Lanka innings. Neglected to account for the wide on the first delivery of the over.
2015-04-17	Bangladesh vs Pakistan	1st ODI	Moved from the 6th to the 8th delivery of the final over in the Bangladesh innings. Neglected to account for the wides on the 4th and 7th deliveries of the over.
2016-08-30	England vs Pakistan	3rd ODI	Moved from the 6th to the 7th delivery of the 37th over in the England innings. Neglected to account for the no-ball on the 3rd delivery of the over.
2017-10-06	Sri Lanka vs Pakistan	2nd Test	Moved from the 2nd to the 3rd delivery of the 90th over in the 3rd innings. Neglected to take the no-ball on the first delivery of the over into account.
2018-06-19	England vs Australia	3rd ODI	Moved from the 4th to the 5th delivery of the 9th over in the England innings. Neglected to account for the wide on the 3rd delivery of the over.
2019-03-10	India vs Australia	4th ODI	Moved from the 3rd to the 6th delivery of the 44th over in the Australia innings. Neglected to account for the wides earlier in the over.
2020-01-24	South Africa vs England	4th Test	Moved from the first ball of the final over to the second (and final) ball of the over (and match). Neglected to account for the wide on the 1st delivery of the over.
2021-03-28	India vs England	3rd ODI	Moved from the 5th to the 7th delivery of the 48th over in the England innings. Neglected to account for wides earlier in the over.

Corrected the team that called for the review

Some reviews had the incorrect team listed as the reviewing team. The following entries have been corrected.

Date	Teams	Event	Reason
2010-01-03	South Africa vs England	3rd Test	Corrected the team that took the review on the 5th ball of the 32nd over in the 3rd innings, from England to South Africa.
2013-01-03	Australia vs Sri Lanka	3rd Test	Corrected the team that took the review on the 6th ball of the 15th over in the 4th innings, from Australia to Sri Lanka.
2017-05-10	West Indies vs Pakistan	3rd Test	Corrected the team that took the review on the final ball of the 31st over in the 2nd innings, from West Indies to Pakistan.
2017-10-29	Zimbabwe vs West Indies	2nd Test	Corrected the team that took the review on the final ball of the 31st over in the 2nd innings, from West Indies to Zimbabwe.
2018-07-12	Sri Lanka vs South Africa	1st Test	Corrected the team that took the review on the 3rd ball of the 6th over in the 3rd innings, from Sri Lanka to South Africa.
2018-10-16	Pakistan vs Australia	2nd Test	Corrected the team that took the review on the final ball of the 44th over in the 3rd innings, from Pakistan to Australia.
2019-12-31	Rangpur Rangers vs Rajshahi Royals	Bangladesh Premier League - T20 - 28th match	Corrected the team that took the review on the first ball of the 5th over in the Rajshahi Royals innings, from Rajshahi Royals to Rangpur Rangers.
2020-01-02	Cumilla Warriors vs Sylhet Thunder	Bangladesh Premier League - T20 - 30th match	Corrected the team that took the review on the first ball of the 5th over in the Sylhet Thunder innings, from Cumilla Warriors to Sylhet Thunder.
2020-03-02	Australia vs New Zealand	ICC Women’s T20 World Cup - 18th match	Corrected the team that took the review on the final ball of the 8th over in the 3rd innings, from New Zealand to Australia.
2020-03-09	Bangladesh vs Zimbabwe	1st T20	Corrected the team that took the review on the 2nd ball of the 6th over in the Bangladesh innings, from Bangladesh to Zimbabwe.
2020-11-08	Zimbabwe vs Pakistan	2nd T20	Corrected the team that took the review on the 4th ball of the 10th over in the Zimbabwe innings, from Pakistan to Zimbabwe.
2021-01-25	Bangladesh vs West Indies	3rd ODI	Corrected the team that took the review on the 4th ball of the 9th over in the Bangladesh innings, from West Indies to Bangladesh.
2021-06-10	Multan Sultans vs Karachi Kings	Pakistan Super League - T20 - 16th match	Corrected the team that took the review on the 4th ball of the 9th over in the Bangladesh innings, from Multan Sultans to Karachi Kings.
2021-06-13	Islamabad United vs Lahore Qalandars	Pakistan Super League - T20 - 20th match	Corrected the team that took the review on the 1st ball of the 20th over in the Islamabad United innings, from Islamabad United to Lahore Qalandars.
2021-06-15	Quetta Gladiators vs Lahore Qalandars	Pakistan Super League - T20 - 23rd match	Corrected the team that took the review on the 4th ball of the 9th over in the Quetta Gladiators innings, from Quetta Gladiators to Lahore Qalandars.
2021-07-25	Birmingham Phoenix vs Manchester Originals	The Hundred Men’s Competition - T20 - 6th match	Corrected the team that took the review on the final ball in the Birmingham Phoenix innings, from Birmingham Phoenix to Manchester Originals.
2021-08-07	Northern Superchargers vs Southern Brave	The Hundred Women’s Competition - T20 - 21st match	Corrected the team that took the review on the first ball of the 6th over in the Northern Superchargers innings, from Northern Superchargers to Southern Brave.
2021-09-19	England vs New Zealand	2nd ODI	Corrected the team that took the review on the 3rd ball of the 32nd over in the England innings, from England to New Zealand.
2021-11-22	Bangladesh vs Pakistan	3rd T20	Corrected the team that took the review on the 2nd ball of the 13th over in the Bangladesh innings, from Bangladesh to Pakistan.

Corrected the decision for the review

The following reviews have had their decision corrected, from “struck down” to “upheld”, or vice-versa.

Date	Teams	Match Type	Reason
2012-04-23	West Indies vs Australia	3rd Test	Corrected the decision for the review on the final ball in the 2nd innings, from “upheld” to “struck down”.
2018-02-21	India vs South Africa	2nd T20	Corrected the decision for the review on the first ball of the match, from “struck down” to “upheld”.
2018-09-23	Bangladesh vs Afghanistan	4th ODI	Corrected the decision for the review on the 3rd ball of the 48th over in the Afghanistan innings, from “upheld” to “struck down”.
2019-08-14	Sri Lanka vs New Zealand	1st Tes	Corrected the decision for the review on the 5th ball of the 78th over in the 4th innings, from “struck down” to “upheld”, as well as removing the umpires call flag.
2020-10-30	Pakistan vs Zimbabwe	1st ODI	Corrected the decision for the review on the 2nd ball of the 44th over in the Pakistan innings, from “struck down” to “upheld”.
2020-11-29	New Zealand vs West Indies	2nd T20	Corrected the decision for the review on the 2nd ball of the 7th over in the New Zealand innings, from “struck down” to “upheld”.
2021-01-14	Sri Lanka vs England	1st Test	Corrected two decisions for reviews, changing both from “struck down” to “upheld”. The changes were on both the 3rd and 5th balls of the 116th over in the 2nd innings.
2021-07-27	Welsh Fire vs Southern Brave	The Hundred Men’s Competition - T20 - 8th match	Corrected the decision for the review on the 4th ball of the 5th over in the Southern Brave innings, from “struck down” to “upheld”.
2021-08-12	India vs England	2nd Test	Corrected the decision for the review on the 6th ball of the 51st over in the 4th innings, from “struck down” to “upheld”.
2021-09-23	New Zealand vs England	4th ODI	Corrected the decision for the review on the final ball of the 22nd over in the New Zealand innings, from “upheld” to “struck down”.

Removed an invalid review

In one case I’d incorrectly applied a review to multiple deliveries. I’ve corrected that so that the review appears just once.

Date	Teams	Event	Reason
2021-06-18	India vs New Zealand	ICC World Test Championship Final	I accidentally added a review to 2 deliveries rather than a single one, so the erroneous entry has been removed from the 6th delivery of the 97th over in the 2nd innings.

Uncontested tosses and Forfeited innings

Sat, 11 Sep 2021 19:45:00 +0000

Yesterday I released the data files for 219 County Championship matches from 2019 and 2021, and also made some additions to the various data formats to accomodate some of the various differences I encountered while processing the data. The new additions to the data formats are related to uncontested tosses and forfeited innings.

From 2016 to 2019 the County Championship allowed the away team to just choose to field at the start of the match rather than having a toss. In order to support this I’ve added an uncontested boolean field to the toss data with in the JSON, YAML, and XML data, and toss_uncontested field within the CSV files. If provided this field will be true in the JSON, XML, or CSV, and yes in the YAML. The other toss fields are still provided so that it’s clear which team “won” the uncontested toss and what the result of that was.

The concept of a forfeited innings has never arisen in over 8,000 matches I’d provided before, so I’ve had to decide how to support this. In order to do this I’ve added a forfeited field for innings in the JSON, YAML, and XML formats which, if provided, will be true in the JSON, or XML, and yes in the YAML files. If an innings is forfeited then no deliveries will be provided.

The XML data format has also received one further additional field, a declared field. While making the other additions I was surprised to find that the XML format didn;t already include it. If provided for an innings it will be true, as with the existing entry in the JSON format.

As a consequence of the addition of the various fields to the different data formats I’ve updated the default version for each format as follows: JSON, from 1.0.0 to 1.1.0; YAML, and XML, 0.91 to 0.92; “Original” CSV, 1.6.0 to 1.7.0, and “Ashwin” CSV, 2.1.0 to 2.2.0. Existing files have not been updated to the new versions as they don’t actually need the additional fields, only the new files will use those version. This shouldn’t cause any difficulties as the new fields are only additions.

Alongside the addition of the new fields I’ve also added some extra possible values for some existing fields. Specifically I’ve added 3 new possible values for the reason match replacements took place; covid_replacement, national_callup and national_release. The first, covid_replacement, indicates that a replacements took place under Covid protocols in place for the match. A national_callup replacement takes place when the replaced player is called up for their national team, and is thus replaced. Finally, national_release indicates that the replacing player has been released from national team duty and is allowed to be inserted into the game.

I intend to continue to add County Championship matches for this season as they take place, and previous years over time, along with looking into add Sheffield Shield matches as well. Any additions to the data formats will be announced as they happen.

Introducing the Cricsheet Register, and a new data format, JSON

Thu, 22 Jul 2021 18:00:00 +0000

Today I’m finally ready to release some of the biggest updates I’ve ever introduced to Cricsheet, along with an accompanying set of smaller changes some of which are that are driven by these updates, and some of which are tweaks for the future. After much rewriting, data validation, experimentation, time, and leg-work I’m finally able to release Cricsheet data in JSON format, along with various additions, and, the Cricsheet Register which provides mappings between the various identifiers used for players and official across various sites.

If you’d like to skip to a particular part feel free, as some changes will interest some people more than others. The sections I’ll be covering here are:

Let’s start with the Cricsheet Register…

The Cricsheet Register

There are two questions I receive more often that any others with regards to players involved in matches. Firstly, would it be possible to add player lists to the match data, so that we can know that a player took part in a match even if they didn’t bat, bowl, or take a catch? Secondly, could I add player ids to the data to make it easier to distinguish between players with similar (or the same) names? Both are good questions, and the additions I’m releasing today take care of both of those issues. I’ll cover the details as they apply to match data files later, however for now let me introduce you to the Cricsheet Register.

What is the Cricsheet Register? It’s a collection of CSV files that provides a unique identifier for players and officials featured in Cricsheet data files, along with details on identifiers used for those people across multiple cricket-related websites/sources. As I write this the Register contains details for 7,190 people, with 10,402 identifiers from various sources. The sources for which I currently provide identifiers are:

Big Bash
Cricbuzz
CricHQ
Cricingif
CricketArchive
ESPNcricinfo
Opta
Pulse

There are 2 files in the Register, people.csv, and names.csv. The main file is people.csv, and contains a single row for each person covered by the data, with a unique identifer assigned by Cricsheet, along with a unique name, and any identifiers from external sites that I’ve used. Every entry in the Register has a Cricinfo identifier, with other identifiers sourced as part of my work on the Cricket Scorecard Accuracy Project.

As an example consider the following row from the data. This entry shows 4 external identifiers for Joe Root (at Cricinfo, Cricingif, CricketArchive, and Pulse), with gaps where we don’t have data. For some of those gaps there won’t be data (for example for a second Cricinfo identifier), but for others it’s just missing right now (such as not having a Cricbuzz identifier due to never having needed it). The identifer (a343262c) is unique to Root, and will be used for him in every file.

a343262c,JE Root,JE Root,,,,303669,,11238,204606,,,,887,

The names.csv file simply contains any extra names I’ve encountered for each person. Generally it will contain the slight variations of names, with initials or full names most common, although there will be different surnames for some (such as when players change names after marrying, or change names for other reasons).

The Cricsheet Register is heavily inspired by the Chadwick Register which does something similar (but more extensive) for baseball.

New default file format - JSON

From today, the default format for Cricsheet data is JSON (with a version of 1.0.0), rather than YAML. All matches are provided in the new JSON format, and as part of that change will include some new data fields. The YAML format isn’t going away anytime soon, so you don’t need to move to JSON right now, however the JSON format does include information that isn’t available in any of the other formats.

I’ll provide a quick summary of the new data available in the JSON, but you can read the full details of the new JSON format, on it’s own page in the Format section of the site.

The main additions

`registry`

With the addition of the Cricsheet Register (covered above) it should come as no surprise that data from it will make an appearance in the new format. Specifically the info section has a registry section which lists all of the people involved in the match, whether players or officials, along with their Cricsheet ID. The names included in the registry are those that will be used for each person throughout the match data file. This means that even if the names of people change it will still be easy to correctly identify them across matches. The details of the registry section are in the documentation for those who are curious so, rather than spoiling that for you, here’s a small example to show how it appears.

"registry": {
  "people": {
    "AJ Finch": "b8d490fd",
    "AJ Turner": "ff1e12a0",
    "AJ Tye": "7c7d63a2",
    "BKG Mendis": "5d1e7582",
    "BR Dunk": "272d796e",
    "CK Kapugedera": "cfad138c",
    "DAS Gunaratne": "770494eb",
    "EMDY Munaweera": "5a22d91c",
    "JA Richardson": "1ee08e9a",
    "JJ Crowe": "2e760301",
    "JP Faulkner": "808f425a",
    "JRMVB Sanjaya": "530b20e3",
    "KMDN Kulasekara": "469ea22b",
    "M Klinger": "b970a03f",
    "MC Henriques": "32198ae0",
    "MW Graham-Smith": "18aca3ce",
    "N Dickwella": "45963d9e",
    "P Wilson": "68304a36",
    "PJ Cummins": "ded9240e",
    "S Prasanna": "f78e7113",
    "SD Fry": "6b725ed1",
    "SJ Nogajski": "9b3f9323",
    "SL Malinga": "a12e1d51",
    "TAM Siriwardana": "bf7842c9",
    "TD Paine": "5748e866",
    "TM Head": "12b610c2",
    "WU Tharanga": "7ed9fd56"
  }
}

`players`

Another edition to the info section is the players field. It lists, for each team, the names of the players officially involved in the match, including the starting eleven, and any supersubs or concussion substitutes. This has been one of the most frequent requests I’ve received over the years so I’m glad to finally release it. An example of the data provided is…

"players": {
  "Australia": [
    "M Klinger",
    "AJ Finch",
    "BR Dunk",
    "MC Henriques",
    "TM Head",
    "AJ Turner",
    "JP Faulkner",
    "TD Paine",
    "PJ Cummins",
    "AJ Tye",
    "JA Richardson"
  ],
  "Sri Lanka": [
    "N Dickwella",
    "WU Tharanga",
    "EMDY Munaweera",
    "BKG Mendis",
    "DAS Gunaratne",
    "TAM Siriwardana",
    "CK Kapugedera",
    "S Prasanna",
    "KMDN Kulasekara",
    "SL Malinga",
    "JRMVB Sanjaya"
  ]
}

`powerplays`

If an innings had any powerplays then there will be a powerplays field in the innings data containing information on each powerplays that took place. All entries how the type of powerplay along with the start and end deliveries for the powerplays. The following example shows the powerplays for an innings with 2 powerplays, one of which was chosen by the batting team:

"powerplays": [
  {
    "from": 0.1,
    "to": 9.6,
    "type": "mandatory"
  },
  {
    "from": 35.1,
    "to": 39.6,
    "type": "batting"
  }
]

`review`

"review": {
  "by": "Pakistan",
  "umpire": "AG Wharf",
  "batter": "Saud Shakeel",
  "decision": "struck down",
  "umpires_call": true
}

`missing`

"missing": [
  "player_of_match",
  {
    "powerplays": {
      "1": [
        "batting"
      ],
      "2": [
        "batting"
      ]
    }
  },
  "reviews"
]

Other changes

A number of smaller additions and changes from the YAML are also appearing in the JSON format, including…

`balls_per_over`

This, unsurprisingly, specifies the number of balls expected within an over in a match. For every match ever published so far by Cricsheet, this is 6. I’m adding this now with a view to supporting The Hundred, and, theoretically, matches of the past should I ever receive ball-by-ball data for them.

`event`

This in an attempt to improve on the existing competition entry in the YAML format, by providing more information and extending it’s presence to matches beyond the domestic T20 competitions. If this field is provided it will always contain a name entry which will be familiar to users of competition. however it’s also possible that the other fields (group, match_number, and stage) will be provided (where relevant). It’s probably best to show this by example. The following is match number 18 of the World T20 in group 1 of the Super 10 stage.

 "event": {
  "group": 1,
  "match_number": 18,
  "name": "World T20",
  "stage": "Super 10"
}

`officials`

This expands on the YAML data’s existing umpires field to also include the names of reverse umpires, tv umpires, and match referees, where I know them. An example of a fully populated entry might be:

"officials": {
  "match_referees": [
    "JJ Crowe"
  ],
  "reserve_umpires": [
    "MW Graham-Smith"
  ],
  "tv_umpires": [
    "P Wilson"
  ],
  "umpires": [
    "SD Fry",
    "SJ Nogajski"
  ]
}

`season`

The season the match was played in, as commonly defined in the cricket world. Right now the season is 2021, whereas last winter it was 2020/21.

Updates to the existing formats

I may be moving to JSON as the new default format for Cricsheet match data but that doesn’t mean that I’m abandoning the existing formats. The YAML, CSV, and XML formats are all receiving updates to include some of the new information included in the JSON files, and in the case of the “Ashwin” CSV receiving a new file for each match.

Addition of Info file for “Ashwin” CSV

I’ve added a new file to contain match information in the “Ashwin” CSV format. The files are named <id>_info.csv (to accompany the existing <id>.csv ball-by-ball files) and provide detailed match information for the match. The existing ball-by-ball files are untouched, and you can use them as you currently do.

The content of the new info files is exactly the same as the info rows in the “Original” CSV format.

General additions

All of the existing formats, along with the new “Ashwin” info files are receiving some additions based on data in the JSON format. Specifically, all of them now have new entries in their info sections, with balls_per_over and registry being added to each, and all also now having player lists for each team (called players in YAML and CSV, and lineups in XML).

The registry and players/lineups additions contain the same data as included in the JSON, with just the structures varying to fit with the existing formats. You can see the details for each of these additions in the relevant Format pages.

Version number changes

With all of these changes it should come as no surprise that the version numbers for each of the existing data formats are changing. YAML and XML are changing from 0.9 to 0.91, “Original” CSV from 1.5.0 to 1.6.0, and “Ashwin” CSV from 2.0.0 to 2.1.0. Someday I may consolidate, and bring consistency to, the various version numbers but that’s not a task for right now!

The future of the YAML format

With the move to JSON as the default format for Cricsheet data it’s time to consider the future of the Cricsheet YAML format. First things first, the YAML format will continue to be provided, without any further additions to the format, for the foreseeable future. It will eventually be phased out, but it won’t be soon, I will give a least 6 months advance warning when I do finally set a date.

I’m also aware of some projects that rely heavily on the YAML data, so I don’t plan to remove it until they have the chance to move to the JSON format. If you feel like you’re one of these feel free to get in touch so that I’m aware of you!

Summary

I hope that these changes will be useful to people, and provide interesting new data to work with. Do get in touch if you notice any strangeness, or issues, as I’ve this involved a substantial amount of work and I’ll be amazed if I’ve carried it out without something going awry.

Hello again

Wed, 17 Apr 2019 19:58:08 +0000

Cricsheet officially went “on hiatus” on the 30th of October 2017, with the last match data actually being added on the 16th of July that year. Since then various factors (some very important, some merely awkward) have kept the site in a state of limbo. You don’t really need (or probably care for) the details, and I’d be reluctant to share anyway, but the important news is that the hiatus is finally over.

I’ve released the data for the matches played so far in the 2019 Indian Premier League (33 matches as I post this) and will continue to do so for the rest of the tournament. I’m hopeful that the data is up to the usual (hopefully high) Cricsheet standards, but would be grateful if people report any errors to me. There have been major changes behind the scenes to how I generate this data and, while precautions have been taken, there is the chance something incorrect will have slipped through.

My short-term plan is to continue adding IPL 2019 matches, while also looking to gradually fill in the gaps in domestic T20 competitions played since the beginning of the hiatus. After that I’ll look to add one-day internationals, before contemplating Test matches. I suspect Test matches will take a while to appear as I took certain shortcuts while working behind the scenes in order to be able to start updating sooner. Deals no with that tech debt will take a little time.

Anyway, hello again. It’s good to be back.

A multitude of updates

Thu, 06 Apr 2017 07:31:44 +0000

It has been over a month since I added new data to the site and, to mark the end of that apparently inactive period, I’m going to provide an update on what I’ve actually been doing, and the results of that work. Make yourself comfortable because this is going to be a longer update.

Big Bash League data

I’ve added data for most matches from the last 2 years of the Big Bash League. This comprises 67 matches, and 15,859 deliveries. There are a few missing matches (you can see which ones on the Missing page, but the majority are there.

I plan to gradually add earlier years for the Big Bash League, and eventually expand to other major domestic T20 competitions, over the next number of months.

Replacement changes

As part of the work to add the Big Bash League I’ve also changed how the data format deals with replacements. Previously replacement information was confined to “super-substitutions” and nothing else. This has now been expanded to cover replacements generally, both at a “match” and “role” level. “match” replacements can be super- or concussion-substitutions (and involve a player replacing another in the match), whereas “role” replacements are occasions where a player replaces another as a bowler or batter (but does not take their place in the match).

This expansion of replacements has resulted in the “replacement” of the “super-subs” field on a delivery with the new and improved “replacements” field. More details on this new field can be found on the Format page.

Version update to 0.9

The expansion of replacements mentioned above has, unsurprisingly, resulted in the data version in the YAML files being changed from 0.8 to 0.9. My plan is that the next update to the data file will change the version to 1.0.0, and that I will stick strictly to semantic versioning from them on.

Name updates

Part of the work I have undertaken in the last month has been to perform a thorough update of the names using for players, and officials in the data files. This has consisted of two real changes. The first was to consolidate some occurrences of multiple names for players into a single instance, while the second update was to ensure that a name is used for only a single player. An example would be Rashid Khan, which was being used for 2 players. I have gone through all such instances and put steps in place to stop them re-occurring.

Innings-level penalty runs - Data version updated to 0.8

Sun, 08 Jan 2017 22:29:02 +0000

47 Test matches were played in 2016 however, as dedicated observers of Cricsheet will have noted, I have, until now, only provided data for 46 of those matches. The 3rd Test of the 2016 New Zealand tour of India was the sole omission. As of today that has been rectified, and we now provide full Test data for 2016.

In that 3rd Test, while batting, Ravindra Jadeja persistently ran on the pitch resulting in 5 penalty runs being awarded to New Zealand before the start of their 1st innings. This method of applying penalty runs was not one that my existing data format could support (as we previously expected penalty runs to be applied on a particular delivery), meaning that I’ve had to apply a small update to the format to support this new development. This has resulted in an update of the data format version from 0.7 to 0.8.

The only change between versions is the addition of an optional penalty_runs field within each innings. If penalty runs were added to an innings, either before or after the innings, then this field will be provided (with pre or post used as appropriate).

Previous data files will be exactly as they were, save for the change of version number, while the newly-added data file for the aforementioned Test match will actually use the new field. If you’ve written code that uses the data I provide you may want to tweak it to take account of the new field.

A long overdue addition - Women's data

Tue, 19 Apr 2016 21:08:58 +0000

I’ve been making data available on this site since 2009 and have gradually increased the number of files I provide to the point that, as I write, 2,780 matches are available. Over time I’ve expanded from just matches involving Full Members, to the Indian Premier League, non-ODI international one-day matches, and international T20s. This gradual expansion means that I’m now providing over 380 matches involving only the Associates and Affiliates, meaning that I’m not just covering the Full Members. This has been an improvement, however there is still one issue, and that is that I’ve only been providing data for Men’s cricket.

I’ve wanted to add data for Women’s cricket for a while. I started the project with the idea of providing cricket data, but I didn’t really think of anything beyond Men’s cricket. Raf Nicholson expressed very well the trap I let myself fall into.

At its heart, it comes down to this: The first “C” in ICC has always, since its formation in 1909, stood for “cricket”, though what it should really have been called, up until it took control of women’s cricket in 2005, was the IMCC - the International Men’s Cricket Council. When a male journalist says, “I am a cricket correspondent”, he means “I am a men’s cricket correspondent.” When a blog refers to itself as an “England cricket blog”, what this generally means is “an England men’s cricket blog”. And when ordinary cricket fans say “cricket”, almost without exception what they really mean is “men’s cricket”. In short, men’s cricket is the default setting. – Raf Nicholson - Why I am a cricket-feminist

I very much fell into the trap of viewing Men’s cricket data as cricket data, and not considering Women’s cricket at all. This is unfair, and something I’ve been planning to fix. Men’s sports have awesome data as Allison McCann has noted, while Women’s sport is poorly served.

And just because the data doesn’t exist doesn’t mean we can’t compile it ourselves or make estimates based on what is available. I just think that in addition to praising the virtues of men’s sports data, we need to acknowledge that good women’s sports data is severely lacking. – Allison McCann - Hey, Nate: There Is No ‘Rich Data’ In Women’s Sports

I’m happy to announce that, as of today, Cricsheet will finally be providing data for Women’s cricket. The initial release consists of 257 matches, comprising 148 T20Is, 69 ODIs, 37 International T20s, and 3 Test Matches, and includes matches from as far back as 2009.

The addition of Women’s data has a practical implication for the data we already provide. The Data Format has just been changed to update the version from 0.6 to 0.7, to allow for the addition of gender as a new field in the info section. Right now this field contains either female or male, but I reserve the right to have other values in the future.

The Downloads page on the site has also been updated to allow users to download Women’s or Men’s matches in all of the variations we previously provided, as well as continuing to download all matches for all genders.

As Raf Nicholson wrote “men’s cricket is the default setting”, and I’ve been guilty of having that mindset. Today is a small step on the path to changing that, and to stop viewing men’s cricket as the default.

Until that changes, we have a problem. Until that changes, I’m going to keep telling the world that I am a feminist. Cricket needs feminism. End of story. – Raf Nicholson - Why I am a cricket-feminist

Version updated to 0.6

Sun, 14 Dec 2014 19:22:20 +0000

The data version included in every data file I provide, and explained on the format page of the site, has just been changed from 0.5 to 0.6. This actually reflects a relatively minor change, and is the first time I’ve bumped the version number since February 2013.

In the 1st Test of 2014 between Pakistan and Australia, Sarfraz Ahmed was dismissed and play stopped for tea. After the break Zulfiqar Babar, who had been batting with Ahmed, didn’t come back out and retired hurt. This meant that in the data I needed to record 2 dismissals related to a single delivery. A complication had arisen.

As I’d never even considered multiple wickets on a single delivery as a possibility, and since it had never occurred in the previous 31,271 wickets I provide data for, I’ve had to tweak the data format, along with numerous scripts, to allow for this possibility. The change I’ve implemented allows the wicket entry on a delivery to contain a list of wickets, rather than always assuming just one. Balls where only a single wicket fell (all 31,271 of them thus far) are unchanged, this tweak simply allows for the possibility of something different.

If you’ve written code that uses the data I provide you should make a small tweak to check for the existence of multiple wickets on a delivery, however, if you don’t, you’ll probably be fine apart from when you try to process that single Test match where this issue.

There will be substantial changes to the data format coming in the next number of months, which will add new information for many of the matches currently covered. These may require tweaks to some of your code, but I will be providing parallel versions of the data files for a period of time, allowing users to continue to use the older version while updating their code. More details on these changes soon.

9 new countries, 91 new matches

Mon, 06 Jan 2014 10:17:30 +0000

3 months ago, in October, we said that we would need to give some further thought as to how we will deal with T20 matches that aren’t regarded by the ICC as ‘T20 Internationals’.

Well we’ve reached a conclusion, and implemented it. We’ve just updated the site with 91 new data files, including 53 international T20 matches (not T20s internationals), and added an extra 9 countries to those we have some data for, namely Denmark, Hong Kong, Italy, Namibia, Nepal, Papua New Guinea, Uganda, United Arab Emirates, and the USA. All 53 of these matches come from the World T20 Qualifier in November 2013. The match_type used for these matches is IT20, meaning “International T20”.

What is the difference between a T20 International and an International T20?

“T20 International” and “International T20” sound like they refer to the same type of match, but there is a subtle difference. A “T20 International” is a match that is recognised by the ICC as being a full international. This means a match between Full Members or those Associates and Affiliates to whom the ICC has “granted” T20 status. An “International T20” is the name we’re using to cover all other international T20 matches, such as those involving a country that hasn’t been granted T20 status.

Confusingly a country can play both types of T20. Ireland did so during the World T20 Qualifiers, playing a “T20 international” against Canada in the group stage while playing “International T20s” in the other group matches.

We don’t think the distinction should exist. If a match is played between any two countries and follows the T20 rules we think it should have the same status as any other similar match. The ICC disagree sadly and we note the difference for accuracy.

What about One-day matches that aren’t ODIs?

We don’t currently have any non-ODI one-day matches on the site. We’ll start adding these when the World Cup Qualifiers start in a weeks time. These will slowly appear on the site with a match_type of ODM.

We will look into adding older data for these types of match too, however finding the data will be the main problem as always. It’s rare that ball-by-ball commentary is provided for these matches, and it’s even more rare for it to be accurate when it is done, sadly. We’re quite good at fixing errors by now (practice will do that) but some sources are so catastrophically bad that we have to just throw our hands up and walk away.

When is an ODI not an ODI?

Tue, 08 Oct 2013 21:56:43 +0000

A person without knowledge of the cricket world when asked to define a “one-day international” might say that it would be a match between two countries played on a single day under an approriate set of rules. This apparently reasonable explanation would be incorrect. This illogical position is currently causing uncertainty as we investigate a forthcoming change of emphasis, and is one of many small signs of the inequality in the world of cricket.

As we write there are 106 nations listed as members of the ICC, 10 Full Members, 37 Associates, and 59 Affiliates. Of those 106 members only the Full Members have the permanent right to play ODIs. 6 of the Associates/Affiliates are “granted” the right to play ODIs for a limited period, depending on how they do in the World Cricket League. The other 90 members don’t get to play ODIs, but they can play one-day matches against other countries (although rarely, if ever, a Full Member).

In June, Andrew Nixon made an observation on Twitter regarding “the lack of variety at the top of international cricket. Same teams playing each other over and over”. We responded that “It’s disturbing how many people have no idea more than 10 countries play the game, and that they don’t see that as a problem”. At that point we realised that we were were falling into a variation of that mistake.

At the moment we provide data for Tests, ODIS, T20 Internationals, and IPL matches. This means that for international cricket we’re focussing exclusively on 16 out of the 106 ICC members. The vast majority of our data files feature only Full Members. We hadn’t even been attempting to include any other international matches. We’ve now decided this must change to include matches featuring all countries. This causes a dilemma for us as we have to decide how to refer to these matches on the site and within the data we provide. It may seem like a minor issue, but it’s one that is standing in the way of this expansion.

We were originally tempted to call all one-day matches between countries ODIs and to be done with it, however we feel that there should be a way to indicate that the matches were viewed as distinct from ODIs at the time they were played. In an ideal world there would be no such distinction but our opinion on this shouldn’t cloud our work in creating reliable data, so that option is off the table. We’re gradually coming to the view that we’ll just call each of these matches a “One-day match” and use the short code of ‘ODM’ for the match_type in the data files. It’s a small difference but it seems to fulfill our requirements.

We haven’t yet implemented this addition in any data file. We still have a number of issues to deal with first, mainly on the website, so that we can adequately distinguish between ODIs and ODMs, but also so that we can have useful coverage figures. We also need to give some further thought as to how we will deal with T20 matches that aren’t regarded by the ICC as ‘T20 Internationals’, as well as dealing with multi-day matches such as the Intercontinental Cup. Once we deal with some, if not all, of these issues we’ll look into adding matches involving the rest of the cricket world to the site. The main problem there will be in finding any source of ball-by-ball data, but we’ll worry about that problem when we can.