The justice secretary, Chris Grayling, is to outline plans for the wholesale outsourcing of the probation service with private companies and voluntary sector organisations to take over the rehabilitation of the majority of offenders by 2015.
The public probation service is to be scaled back and "refocused" to specialise in dealing only with the most dangerous and high-risk offenders and public protection cases. The majority of services will be contracted out on a payment-by -result basis.
The probation officers' union, Napo, claims the move represents the demise of the 105-year-old public probation service in England and Wales.
Grayling says a radical overhaul is needed to tackle the high reoffending rates with 58% of short-sentenced prisoners reoffending within a year and half a million crimes committed each year by released prisoners.
The plans published on Wednesday mark a rapid acceleration of the "rehabilitation revolution" that the previous justice secretary, Kenneth Clarke, had been cautiously piloting.
Harry Fletcher, an assistant general secretary at Napo, claimed the decision was astonishing given that the service had been awarded the British Quality Foundation Gold Award for Excellence last year: "This move is purely ideological. It is being rushed through without proper thought to the consequences. It will be chaotic and will compromise public protection."
The plans set out in the consultation, Transforming Rehabilitation, envisage inviting private companies and voluntary sector organisations to bid for the overwhelming majority of probation work. Full implementation is envisaged to take place by spring 2015, just before the expected date of the next general election.
The announcement by the rightwing, populist justice secretary represents a new frontier in the boundary between public and private sectors in criminal justice. More than 240,000 offenders are supervised by the probation service each year.
Unions expect that as much as 70% of the work currently done by the probation service in working with offenders in the community will move to private and voluntary sector providers under the plan, including the supervision of nearly all medium and low-level risk offenders.
While the public probation service will not be banned outright from bidding for the work, it will be expected to do so only in partnership with the private sector. The current arrangement in London where Serco and the probation service delivers the community payback or unpaid work contract is regarded as the most likely model.
The first phase to be subject to competition will centre on Grayling's plans to introduce mandatory rehabilitation programmes for short-sentenced prisoners on their release. Legislation is to be introduced within the next year which will require the 46,000 offenders serving sentences of less than 12 months to undertake drug treatment, mentoring or other programmes as a condition of their release.
The consultation paper says the remaining role of the public sector probation service "will focus on protecting the public by managing the most high-risk offenders, including all serious sexual and violent offenders, providing advice to courts and making initial risk assessments on all offenders. It will retain ultimate responsibility for public protection in all cases". High risk offenders account for only 51,500 of the 240,000 under probation supervision each year.
The decision to leave the writing of court reports to the probation service reflects the potential conflict of interest if private security companies are recommending sentencing options to judges and magistrates. The retention of the ultimate responsibility for public protection implies that the probation service will also keep an important monitoring role to safeguard standards and avoid major scandals.
The structure of the probation service also faces a major shakeup with the current 35 areas merged into possibly six to eight regions so that bids can be invited on a national commissioning basis. The new areas will align closely with those of local authorities and police and crime commissioners. A £500,000 fund is to be set up to ensure voluntary and community groups are ready to begin bidding for services.
"Private and voluntary sector organisations will then be invited to bid for work in these areas with each contract awarded based on best value and innovation in tackling offending," says the consultation paper.
Grayling said the plans were the most significant reforms to tackle reoffending and managing offenders in the community for a generation.
"What we do at the moment is send people out of prison with £46 in their pocket, and no support at all. No wonder we have such high levels of reoffending. It is madness to carry on with the same old system and hope for a different result," said the justice secretary.
"We know across the public, private and voluntary sectors there is a wealth of expertise and experience – we need to unlock that so we can finally begin to bring down our stubbornly high reoffending rates.
"Our proposals will see all of those sentenced to prison or probation properly punished while being helped to turn away from crime for good. They will also mean we only spend taxpayers' money on what works when it comes to cutting crime," he said.
But Fletcher said the claim that high reoffending rates among short-term prisoners was evidence of probation failure was unfounded: "Probation has no statutory responsibility for supervising anybody sentenced to 12 months or less. Reoffending rates for the individuals that Probation does supervise are much improved; those who participate in programmes have a reoffending rate now of 35%. This is a success story that the government should be building on, not destroying", he said.