Luminact Insights by Kaylene Askew – 20 September 2022
Calculating the cost of delivering a contract with Defence can be tricky. Variables that influence the actual estimates for a specific scope of work vary, but they usually boil down to performance, compliance, and the quality of the estimation of the effort itself.
There is however one part of ASDEFCON (Australian Standard for Defence Contracting) contracts that can result in either a significant over or under-estimate when not addressed correctly. This is the cost of delivering documentation that is specified by the Contract Data Requirements List (CDRL).
On face value this annex to the Statement of Work (SoW) is simply a list of documents to be delivered in alignment with the product development, delivery, and ultimately acceptance. But for the inexperienced ASDEFCON bidder, this is the tip of the iceberg that could sink the contract ship.
This article looks at the cost of compliance to the CDRL and why paying special attention to this part of a contract is so important. To help the reader explore some of these issues we created a simple CDRL cost model against the ASDEFCON (Complex Materiel) Volume 2 [1] suite of tendering documents, which we then used to demonstrate why this needs attention.
For those not bidding on Defence contracts, this analysis will help illustrate why military procurements quickly add up to millions (or billions) of dollars, and how a significant portion of this relates to documentation rather than actual hardware or software.
Back in 2016 a newspaper article summarised a customer’s restaurant review which related to a complaint regarding the cost of a glass of hot water with a slice of lemon [2] which was priced at £2. The restaurant managers’ response to the review, which subsequently divided readers, explained the sheer number of steps and energy involved in serving a glass of hot water with a slice of lemon.
The division of the readers came down to a simple question: “How much do you think you should have to pay for a glass of hot water with a slice of lemon at a café?” The newspaper article described that the customer’s review indicated that the £2 price tag was “too expensive” and the “staff were rude”. Setting aside any opinions on the staff performance, what was your initial response to the query of how much you were willing to pay?
If you have previously worked in a busy café, you may have a better understanding of what the true cost of an order like this involves. Or like most, you may believe that despite these factors the cost to the café is negligible and the price is not worth charging anyhow.
The restaurant’s response to the customer review was detailed, explaining the process of showing someone to a table, taking orders, sending the chit sheet to the kitchen, preparing the drink, serving, rounding out the bill, clearing and cleaning the table and the opportunity cost of a customer who may have sat at the same table and ordered a meal. In fact, the explanation resulted in a calculation (including electricity and staff time) that cost more than £2.
Now imagine all your customers come in and order a glass of hot water with a slice of lemon and consider how you feel if you had to repeatedly cover these losses through other sales above the charge rate. This can add up quickly.
It was shortly after reading this article that I realised this is analogous to discussing Defence contract deliverables. Experienced defence organisations understand the cost of documentation requirements well, especially when there are rejections or revisions involved (and not always associated with the cost of non-quality), and schedule delays kick-in. However, if you have never experienced these costs before it can be difficult to imagine how they can stack up so quickly.
Let’s work through an example based on the standard, unmodified ASDEFCON (Complex Materiel). We have built a cost model identifying each deliverable over time factoring in the time-based requirements and the event-based requirements such as Mandated System Reviews (MSRs). Let us be clear that we only factored in a minimalistic scenario, based on the following assumptions:
- A single product (and therefore product baseline) through the MSR process, and one System Acceptance (SA) event.
- Data items that often have multiple documents incorporated (i.e., design documentation, drawings, and publications) are counted as one delivery.
- A 0% rejection rate (in reality this is extremely optimistic)
- The end-to-end contract duration from Effective Dated (ED) to Final Acceptance (FA) is 24 months.
Based on these factors alone the results are:
- 73 unique data items line items are to be delivered
- 303 deliverables for these line items occur over the 24-month period
- Rounding down, this equates to an average of 12 data items delivered each month over a 20-day working month which equates to more than one item for every two working days
Sound confronting yet? Now we will look at the effort required. Let us make some assumptions related to staff members:
- A person will work eight hours per day, and a working week is 40 hours, resulting in a working month being 160 hours
- A working year is only 220 days as this will factor in sick leave (ten working days), annual leave (20 working days) and public holidays
- A working year therefore equates to 1,760 hours, and one Full Time Equivalent (FTE) staff member
The first thing we need to do once we have established our effort assumptions is to identify the ratio related to the process of developing, reviewing and approving a document. We need to establish a ratio for our estimate which for the purposes of this example we have identified as:
8 (Author) : 2 (Reviewer) : 1 (Approver)
Which means that for every eight hours of authoring, two hours are required to review and one hour is required for an internal approval. If we then look at the hours of effort that may apply to the authoring alone and assume that on average a document will take 80 hours to author, then using the ratios this will result in a further 20 hours of review and ten hours to approve each deliverable.
Now at this stage one can argue that after the initial delivery, the effort to update and maintain a document will decrease with each delivery cycle required. Technically this should be the case, however in reality if you are developing a document from scratch, the first instance generally takes you much longer than 80 hours to author. In addition, the effort for some deliverables will increase as the volume of data generated increases with the design process (drawing, publications etc.). So, for this example these figures are a reasonable average as some documents will take months and others will take five days (40 hours). To be conservative we have applied the effort per delivery as:
- Author = 80 hours per delivery
- Reviewer = 20 hours per delivery
- Approver = 10 per delivery
- A total of 110 hours per delivery
Using this information, let us now look at what these figures tell us when we apply these values to 303 deliveries:
- This equates to 33,330 hours of effort
- When you divide 33,330 by 1,760 it equates to roughly 19 FTE
- If you spread this over two years this is roughly 9.5 FTE per year developing documentation alone
Following this we can apply some hourly rates and surmise what this burden might look like in AUD$. For this example, we will apply the following hourly rates:
- Author $80 per hour
- Reviewer $100 per hour
- Approver $120 per hour
The calculations of the effort in hours multiplied by the rates equates to an embedded cost of:
$2.9m to comply with the CDRL
Some people may think that once you have an approval for a document that you can just submit a maintenance letter and not worry about updating your document. Once again, this may be true however in the above calculations we haven’t factored in overheads for quality assurance, software, data management, configuration management and the additional cost of releasing controlled documents with a cover letter explaining why you’re delivering a document let alone the cost of developing and delivering the product or service required under the contract.
There is no doubt there is a division in the readers responses to these values. Some will not believe these figures, and some will be thinking ‘you will never win a bid with these costs embedded’, others will be thinking about the fact that this is clearly already an underestimation.
So, just like the glass of hot water with a slice of lemon, there is a perspective that needs to be applied. If your customer is ordering top shelf menu items and glass of hot water with lemon on the side, it might be ok that they expect not to have to pay, however if your customer is ordering a glass of hot water with an entrée where the profit on the entrée is less than £2 then perhaps you need to rethink how you approach this customer. Perhaps you need to reshape their thinking and provide a more ‘efficient’ document delivery schedule against their requirements.
Do not ignore the cost of generating documentation for ASDEFCON compliance. The burden of the deliverables is real, and rejections have the added burden of delaying your schedule. This in turn will add additional time phased document deliveries. The figures provided are conservative for those new to ASDEFCON, we encourage organisations to do their own calculations and find out how much a glass of hot water with lemon will actually cost your company to deliver?
References
- https://www.defence.gov.au/business-industry/procurement/contracting-templates/asdefcon-suite/complex-materiel-vol-2
- https://www.theguardian.com/lifeandstyle/wordofmouth/2016/jan/14/is-2-a-fair-price-for-a-cup-of-hot-water-and-lemon
