|
7. TECHNOLOGY PROCUREMENT
PROCESS (RESPONSE TO MOTION 78/5) PROCESSUS
D’APPROVISIONNEMENT EN MATIÈRE DE TECHNOLOGIE (RÉPONSE À LA MOTION 78/5) |
That Council approve this report with the following amendment:
·
That any change to suppliers’ pre-qualified teams undergoes a
requalification process as rigorous as the initial qualification process.
Recommandation du ComitÉ
Que le Conseil approuve ce rapport contenant la
modification suivante:
·
Que tout changement apporté aux équipes
pré-qualifiées de fournisseurs entraîne un processus de requalification aussi
rigoureux que le processus original.
Documentation
1.
City
Treasurer’s report City Operations
dated 10 May 2010 (ACS2010-CMR-FIN-0039).
2.
Information
Technology Sub-committee Extract of Draft Minutes of 17 May 2010.
Information Technology
Sub-Committee
Sous-comité de la technologie de l’information
and / et au
Corporate
Services and Economic Development Committee
Comité des services organisationnels et du développement économique
and Council / et au Conseil
Submitted by/Soumis par :
Marian Simulik, City Treasurer/Trésorière municipale
Contact
Person/Personne ressource : Jeff Byrne, Manager, Supply / Gestionnaire, Approvisionnement
Finance
Department/Service des finances
(613)
580-2424 x25175, Jeff.Byrne@ottawa.ca
|
Ref
N°: ACS2010-CMR-FIN-0039 |
SUBJECT:
|
|
|
|
|
OBJET :
|
PROCESSUS D’APPROVISIONNEMENT EN MATIÈRE DE
TECHNOLOGIE (RÉPONSE À LA MOTION 78/5) |
That the Information Technology Sub
Committee receive this report and forward it to Corporate Services and Economic Development Committee
and Council for information.
Que le Sous-comité de la technologie de
l’information dépose le présent rapport et le transmette au Comité des services
organisationnels et du développement économique et au Conseil à titre
d’information.
Although a comprehensive legal
review of the Next Stop Announcement System (NSAS) procurement found no fault
with the contracting process, there are always improvements that can be made,
and lessons to be learned.
As such, a working group chaired by Supply
Management, with representation from Legal Services, Information Technology and
Transit Services developed a Lessons Learned report in response
to Council Motion 78/5.
MOTION
NO. 78/5 - Smartbus Features NSAS - That a “lessons learned” report be prepared
on the overall technology procurement processes and be submitted to the IT
Sub-Committee for recommendation to the Corporate Services and Economic
Development Committee for ultimate recommendations to Council
As discussed in report No. ACS2009-ICS-TRA-0011, ADVANCING SMARTBUS FEATURES, the requirement for a next-stop announcement
system (NSAS) was brought to the forefront by a 2007 order by the Ontario
Human’s Rights Commission to the Toronto Transit Commission (TTC) to have all
stops announced on board their vehicles.
The City of Ottawa received a similar order through the Canadian
Transportation Agency (CTA).
Automatic next stop announcement is one of many communication components
that would improve the customers’ ability to confidently navigate through the
transit system and throughout the City.
As specified in the solicitation documents, the NSAS procurement
strategy followed a three-stage framework.
Stages 1 and 2 involved a Request For Qualification (RFQ) from potential
proponents outlining their proposed solutions and technology.
Participation in Stage 3, which consisted of a Request For Proposal
(RFP) process, was limited to proponents pre-qualified in Stages 1 and 2
respectively.
Four proponents responded to the City’s RFQ. Two proponents were dismissed as
non-compliant to the qualification requirements. Two proponents, Bell and Clever Devices were
given the opportunity to submit proposals through an RFP. One of the two proponents, Bell, was
disqualified at this stage after presenting an alternate unqualified
partner/vendor in their RFP submission.
Following the disqualification of Bell in early April 2009, Clever Devices' proposal was evaluated and found to be fully compliant.
Report No. ACS2009-ICS-TRA-0011, ADVANCING SMARTBUS FEATURES, was tabled with the Transit Committee on September 16, 2009 to obtain approval to reallocate funds between already approved budget line items. It was during the budget reallocation process that the City received a substantive objection that necessitated that the contract award be made by Council.
On November 16, 2009 City Council approved the award of a $12.1M contract to Clever Devices.
All of the recommendations in this report have been implemented or are in the process of being implemented.
In support of
Motion 78/5 a working group chaired by Supply Management, with representation
from Legal Services, Information Technology and Transit Services, was
assembled. This Lessons Learned report summarizes the findings of this working
group.
Project Management Framework
For large, sophisticated information technology projects it is a best
practice to establish a project management steering committee at the outset of
the project. This practice has been
implemented successfully within the City on other major procurements such as
Pay & Display Parking, Automated Water Meters, P3 projects, and many
others.
The Smartbus program is comprised of many separate, yet interrelated,
projects formally managed by Transit Services. At
the time of the NSAS procurement, Smartbus projects were managed, approved and
procured at the individual project level and the Smartbus program as a whole
lacked integration.
The establishment
of a Steering Committee to oversee the SmartBus Next-Generation initiative, as
well as the identification of the GM Transit as executive sponsor, solved the
project management authority issues associated with Smartbus.
For large, sophisticated, information technology projects ensure that a project management framework and steering committee is established at the outset of the project, and maintained throughout the life of the initiative.
As part of the larger Smartbus initiative, Bell was the successful proponent in a 2003 competitive RFP for the provision of a wireless data network solution and intelligent transit vehicle subsystem. Included with these deliverables was the requirement for a platform on which the required systems would reside.
The Smartbus
platform provided by Bell was based on a closed architecture. In other words,
the platform was limited to applications designed and proprietary to Bell. As
indicated above, Smartbus is not a
stand-alone solution and requires a platform that has a number of integrated
technology applications.
Should Bell have been the successful proponent
of the NSAS solicitation, the Bell solution would have resided on the existing
Bell platform. However, due to the closed architecture of the Bell platform,
were Bell to be unsuccessful a second platform would be required to house the
NSAS.
Although Staff considered the installation of
a second platform acceptable, the
extent to which new technology can be added to the system is reliant on the
capability of the platform.
As Smartbus was made up of smaller interrelated projects, many of which
were considered future initiatives, the acceptability of a closed-architecture
solution should have been considered prior to the 2003 wireless
data network solution and intelligent transit vehicle subsystem RFP.
For large information technology procurements, ensure that the acceptability of closed architecture systems is reviewed in light of the potential impact on future/planned initiatives prior to any solicitation.
As indicated above, it was Staff’s expectation that should Bell have been the successful proponent
of the NSAS solicitation, the Bell solution would have resided on the existing
Bell platform. It was only after the NSAS RFP that Bell advised of impending
lifecycle issues associated with its existing systems that would require a
significant investment to upgrade the
closed-architecture platform to enable the NSAS functionality.
It was not known to Staff at the time of the RFP that the Bell system would require an additional investment. These required lifecycle investments were not included in Bell’s RFQ/RFP response. Had Staff been aware of these impending lifecycle issues earlier in the procurement process, they could have been accounted for in the project budget and solicitation documents.
For large information technology procurements, ensure that a lifecycle review of existing and integral infrastructure has been completed in advance of the procurement process, and the findings included in the solicitation documents.
As specified in the solicitation documents, the NSAS procurement
strategy followed a three-stage framework taking place over 2008 and 2009. Stages 1 and 2 involved a Request For
Qualification (RFQ) from potential proponents outlining their proposed
solutions and technology.
Participation in Stage 3, which consisted of a Request For Proposal
(RFP) process, was limited to proponents pre-qualified in Stages 1 and 2
respectively.
In accordance with the Purchasing By-law, the
RFQ was posted publicly on Merx; however, for projects such as NSAS where
specialized expertise is required and it is expected that the number of firms
capable of providing the required expertise is limited, Staff should review whether
posting on Merx alone is sufficient.
As the
specialized expertise required may be called upon infrequently, perspective
suppliers may not be as vigilant in keeping apprised of potential
opportunities. In order to ensure that competition is maximized, and that the best
potential proponents respond to the City’s solicitations, Staff should consider
providing advance notification of upcoming opportunities through methods other
then Merx.
Lesson Learned
For
large, sophisticated, information technology projects where specialized
expertise is required, Staff should review the best means of engaging the
supplier community.
With regards to the NSAS procurement, Bell and Clever Devices were
pre-qualified in Stages 1 and 2, and were given the opportunity to submit
proposals in response to an RFP. However, Bell’s proposal included the
substitution of a key team member pre-qualified as part of the RFQ. This
substitution was in violation of the terms and conditions of the RFP and
resulted in Bell’s disqualification.
Considering the length of the procurement process (three stages taking
place over two years), Staff should review the acceptability of permitting
suppliers to propose changes to their pre-qualified teams in their RFP
responses.
Substitutions are not without risk; and as such an acceptability review
would provide Staff with an opportunity to consider risk mitigation techniques,
or where substitutions would be unacceptable, support should this determination
ever be called into question.
When multi-stage procurement processes are proposed over an extended period of time, Staff should review the acceptability of permitting suppliers to propose changes to their pre-qualified teams in their RFP responses.
In order to ensure
maximum competition, Supply Management has implemented a Procurement Plan
Approval process whereby requirements and strategies are reviewed and approved
in advance of a procurement process. This approval process is aimed at ensuring
that requirements are not unduly restrictive, and that multi-stage procurement
processes do not unnecessarily limit competition.
Lesson Learned
In order to ensure maximum competition, Staff should complete a review of proposed requirements and procurement strategies in advance of the procurement process. This review should be aimed at ensuring that requirements are not unduly restrictive, and that multi-stage procurement processes do not unnecessarily limit competition.
As part of the RFQ
process, proponents were encouraged to propose as options value added features
and functionality including, but not limited to:
· Location-based advertising
· Displaying other passenger information (special events)
· News, weather, sports
· Amber alerts/public safety
· Other multimedia opportunities
· Premium or subscription services, e.g. Wireless Internet access
· Services-based offering, i.e. little or no initial capital outlay, with the system supported entirely by a vendor, with annual operating charges over a fixed lifecycle or term.
Following an
evaluation of the RFQ submissions, it was determined that only one of the
pre-qualified proponents, Clever Devices, could propose meaningful value added
functionalities.
Following the
disqualification of Bell’s RFP response, Clever Devices' proposal was evaluated
and found to be fully compliant. A subsequent Best and Final Offer (BAFO)
process resulted in a pricing structure of $12.1 million for core system
requirements and an additional $5M for optional requirements
The inclusion of these value added features became a source of contention in seeking funding and contract approval.
When it was determined that only one proponent was capable of proposing optional value added functionalities, Staff should have removed these optional features from the RFP process. If still desirable, the value added features should be sought through a separate solicitation.
Where optional value added features are sought as part of a multi-stage procurement process, Staff should review the appropriateness of including these features based on the information obtained in the preliminary phases of the procurement. Where their inclusion is deemed appropriate, these features should be fully priced as part of the financial proposal.
Approval Process
As indicated above, the Smartbus program is
comprised of many separate, yet interrelated, projects. Although under the
larger Smartbus umbrella, budget approval was sought on a project-by-project
basis. As a result, the estimated cost of the Smartbus initiative remains
undefined, and future identified Smartbus related projects remain unfunded (ex.
real-time bus
arrival, Cameras and Automated Vehicle Maintenance).
With regards to the NSAS, this project-by-project approval format
created confusion when attempting to reallocate funding between Smartbus
projects.
As described above, the Clever Devices proposal included $5M in value
added features. Report No. ACS2009-ICS-TRA-0011,
ADVANCING SMARTBUS FEATURES, advised that “an existing 2009 Capital Project for Transit Vehicle Information System was
approved to add an on-board router to manage the multiple upcoming features of
the SmartBus program.
The Clever Devices added-value options will deliver equivalent
technology; therefore, the Capital Budget of $5.65M assigned to this project
will be used to acquire the Clever Devices’ added-value features, which is
within existing authority.”
Although all Smartbus projects, the redistribution of funds between
projects was complicated by the fact that the funding had been approved for
each project individually. Had Staff sought approval of the Smartbus initiative
as a whole the redistribution of funds within interrelated projects may have
been more easily understood and accepted.
Moreover, when seeking the upfront approval an outline the proposed
procurement strategy and timing for the entire initiative, including each
sub-project, should be provided.
As effectively communicating the short, medium and long term plans
associated with complicated IT projects can be a challenge, care needs to be
taken to ensure that key messages are not lost. This initial procurement plan
could be used as a key communications tool when approaching Committee and
Council.
In addition, as things change (technology, schedule, proponents) the
initial procurement plan could be revisited and revised as a communications
exercise.
For large sophisticated initiatives comprised of multiple sub-projects, Staff should:
·
Seek budget approval for the
entire initiative upfront (as opposed to seeking funding on a sub-project
basis);
·
Develop a procurement plan
as a communications tool identifying the timing and proposed strategy for
procuring the components comprising the initiative as a whole;
·
Provide regular updates to
Committee and/or Council on significant changes (technology, schedule, proponents) and their
impact on the procurement plan.
Although a comprehensive legal review of the NSAS procurement found no fault with the contracting process, there are always improvements that can be made, and lessons to be learned.
All of the recommendations in this report have been implemented or are in the process of being implemented.
As a best practice, Staff intends to conduct lessons learned analyses such as this at the conclusion of all high profile, complex, procurements going forward.
There are no rural implications.
There are no
legal / risk management impediments to implementing any of the recommendations
in this report.
There are no financial implications.
The
Supply Management Branch will apply the lessons learned to future procurements
and conduct a lessons learned analysis at the conclusion of all high profile,
complex, procurements going forward.
|
information
technology sub-committee extract
of draft Minutes 7 17
may 2010 |
|
sous-Comité de la
technologie de l’information extrait de l’ébauche du
Procès-verbal 7 - le 17 mai 2010 |
CITY
TREASURER
TRÉSORIÈRE
MUNICIPALE
FINANCE DEPARTMENT
SERVICE DES FINANCES
TECHNOLOGY PROCUREMENT PROCESS (RESPONSE TO MOTION 78/5)
PROCESSUS D’APPROVISIONNEMENT EN MATIÈRE DE TECHNOLOGIE (RÉPONSE À LA
MOTION 78/5)
ACS2010-CMR-FIN-0039 City Wide/à l'échelle de la Ville
Jeff Byrne, Manager, Supply, Finance provided a
PowerPoint presentation, a copy of which is held on file. Mr. Byrne explained that a comprehensive legal review of
the Next Stop Announcement System (NSAS) procurement found no fault with the
contracting process, but did find opportunities for improvements. The presentation outlined lessons learned in
the areas of project management framework, operational requirements, the
procurement process, and the approvals process.
All of the recommendations in the report have been implemented or are in
the process of being implemented.
Councillor Legendre referred to
the finding that when multi-stage procurement processes are proposed over an
extended period of time, staff should review the acceptability of permitting suppliers
to propose changes to their pre-qualified teams in their RFP responses. He did not agree with the proposition that
Bell should have been permitted to do this and he inquired whether that was
truly staff’s belief. Mr. Byrne replied
that staff is not suggesting that Bell should have been able to replace a key
team member. Rather, they are proposing
that flexibility could be built in for lengthy procurements going forward that
would allow member substitutions upon Council approval. He noted the City uses a replacement of
personnel clause for its contracts, which allows replacement of team members
provided they are of similar ability and subject to Council approval.
Councillor Legendre questioned
whether the issue with Bell was that the replacement member the company had
tried to bring on board was not of the same acknowledged capacity. Mr. Byrne replied that staff did not evaluate
the substitute’s abilities because the rules stipulated that substitutions would
not be permissible. Councillor Legendre
suggested there might be an issue with substitutions if there aren’t enough
qualified people to draw from within a specified field.
Chair Wilkinson suggested there are an ample number of equally qualified
people in the fields in question and she thought it should not be a
problem. She went on to question the
criteria around pre-qualification of suppliers and she inquired if staff have
examined whether the criteria the City is using is too restrictive.
Mr. Byrne indicated staff did consider that and the result is the Procurement Plan Approval process, whereby
requirements and strategies are reviewed and approved in advance of a
procurement process. This will allow for
maximum competition and unnecessary elimination of too many suppliers.
With respect to pricing value added features
as part of the financial proposal, the Chair inquired whether one set price
would be put forward or if the cost of individual enhancements would be noted
within so there would be an option to pick and choose. Mr. Byrne confirmed the latter and further
verified that a company that does not include enhancements in its bid could
still qualify for the basic project.
Chair Wilkinson remarked that many of the
lessons learned statements referred to ‘large’ IT procurements and she
questioned the definition of large in this case. Mr. Byrne responded that where there is a
history of procurement for common technology, e.g. cell phones, he would not
suggest there is a need to bring that to the attention of Committee and
Council, as opposed to procurements of new technologies or for significant
projects.
Councillor Legendre proposed that the report recommendation be amended
so that the Sub-committee could endorse the report and recommend adoption of
the strategies contained within, rather than simply receiving it. He also felt that with respect to personnel
substitutions within pre-qualified teams, any such change should be subject to
a qualification process equal to that of the initial pre-qualification
process. Mr. Byrne agreed.
Moved by J. Legendre:
That the
recommendation for this item be amended to read “approve” rather than “receive”
and that the report be amended to clarify that any change in pre-qualified
‘teams’ undergo a requalification process as rigorous as the initial qualification
process.
CARRIED
The committee then considered the
report recommendation as amended.
That the
Information Technology Sub Committee approve this report and forward it
to Corporate Services and Economic Development Committee and Council for approval,
with the following amendment:
·
that any change to suppliers’ pre-qualified teams undergoes a
requalification process as rigorous as the initial qualification process.
CARRIED as amended