As we reported a fortnight ago, the foundations of New Zealand’s farm environment management system were shaken by the release of a report into the effectiveness of farm nutrient modelling system Overseer.
The report by an independent Science Advisory Panel identified shortcomings with the current version of the nutrient modelling software and concluded it had no confidence in Overseer’s ability to estimate nitrogen losses from farms in its current form.
Farm leaders expressed their dismay but Environment Minister David Parker and Agriculture Minister Damien O’Connor promised the Government will help develop improved tools to manage and estimate total on-farm nutrient loss.
The Ministers welcomed the government-appointed Panel’s report and said it will help develop improved tools for farmers and regulators to meet future Essential Freshwater planning requirements.
The Science Media Centre asked experts to comment on what options might be available for farm nutrient management in the future.
Here are the responses –
Professor Louis Schipper, environmental biogeochemist, University of Waikato:
“There isn’t any simple easy statement about how to progress immediately and longer-term but we critically need a tool like Overseer to allow us to balance production and its impacts on the environment. Otherwise, we might end up regulating inputs of nutrients and potentially decreasing flexibility that farmers might have to manage their land. The tool needs to have the confidence of farmers and the public and be supported with hard data. We have some existing data collected through time but this will need to be supplemented with extensive research that helps the engine of any model work better across broad environments.
“The review identified issues with the underlying setup of the model and its inner workings with one focus being how nitrogen cycling is captured. We can name many of the steps of the nitrogen cycling involved and have a general understanding of the factors that control these rates. If we want to manage nitrogen losses from farming these processes need to be well captured whether developing a new model or adapting Overseer. However, I suspect that we simply do not know how fast or slow some of these nitrogen processes are going in soil across the varying farming environments of our country. Some processes are very well known like nitrogen fertilizer inputs and nitrogen export in milk but others like denitrification (conversion of nitrate to nitrogen gas) or immobilisation are extremely poorly quantified. This requires a greater quantification of basic soil processes which is not a large part of our current research agenda.
“Simply importing an overseas model is unlikely to be very useful without extensive modification to fit our rather unique farming systems and underpinned by data from Aotearoa New Zealand. However, the internal structures of these international models may allow us to make more rapid progress if underpinned by careful soil and water research on our farms.”
No conflict of interest declared.
Professor Troy Baisden, environmental scientist, University of Waikato; and President of the New Zealand Association of Scientists:
This comment is excerpted from Professor Baisden’s blog.
“An MPI-commissioned peer review report has concluded that key aspects of the Overseer model used for water quality regulation are not fit for purpose. The Parliamentary Commissioner for the Environment (PCE) has described the report as “devastating”, while those responsible for the model defend it to users.
“The contrast confirms an undesirable situation I’ve previously framed by saying “Overseer is the best model we have, because it is the only model we have.” The problem evolves from a paradox those who develop scientific models have been aware of. Models that are heavily used can be very difficult to improve and modify as needs change. A model’s usability can be its own enemy when it comes to incorporating detailed science. And it is important to note that the review didn’t examine the usability and value of the Overseer user interface, and farm data collection.
“Those looking at the situation outlined by the peer review panel’s report on the Overseer model last week, will be wondering how such an array of problems and challenges can arise. Overseer has become essential to New Zealand’s water quality regulation with an emphasis on regions and catchments where nitrate problems exist. Overseer was designed for these environments: well drained soils over aquifers. By becoming a one-size fits all model supporting regulation, the Overseer model has captured focus for the entire country, including areas and situations the review finds are outside its core strengths and stated purpose.
“Can the “next generation Overseer” overcome the paradox that its dominant position, large user base and interface makes needed model improvements challenging? Achieving this, or other approaches that quickly reach the scale and quality needed will be difficult. Given the urgency, some may say it will be risky and duplicative to allow multiple approaches to replace Overseer. I argue the diversity of principles and spatial scales (from farms to regional and national policy), and timeframes of need establish different targets and necessitate development of some complementary and overlapping tools. We should beware of ever again placing so much weight on a single tool.”
No conflict of interest declared.
Dr James Lockhart, Senior Lecturer, School of Management, Massey University:
“One of the root causes of this problem is that science typically makes marginal/incremental contributions. Thomas Kuhn challenged what has emerged as ‘normal science’ in his book, “The Structure of Scientific Revolutions”. Kuhn, arguably one of the most influential philosophers of modern science, pointed out that most science operates to extend an existing paradigm by the pursuit of filling gaps in knowledge.
“Hence with ‘Overseer’ value is seen to be created by making it better, and increasingly better over time. That the fundamental assumptions may (or may not) be flawed is unlikely to be questioned. I think the defence of Overseer by the CEO was valid – it is now being used for purposes for which it was not designed. Worse there is a pursuit of national standards, rules to which the individual farmers are being expected to comply – which makes Overseer’s task all the more improbable to succeed. This then creates a sunk cost problem, making change all the more difficult.”
Conflict of interest statement: “I am a user/subscriber to Overseer in Horizons (Manawatu Wanganui Regional Council)”
Dr Scott Larned, Chief Scientist – Freshwater & Estuaries, NIWA:
“NIWA welcomes the independent science review report on the OverseerFM model.
“The review answered several important questions we and others have had about the model (including model structure and prediction uncertainty) and its suitability for specific applications, notably limit-setting and assessing compliance with regional rules and consent conditions.
“The use of Overseer for regulatory purposes has grown rapidly over the last 10 years, largely in response to the requirements of successive National Policy Statements for Freshwater Management and the absence of alternate tools. NIWA concurs with the conclusion of the Science Advisory Panel, that Overseer is not suitable for regulatory purposes such as limit setting or demonstrating compliance. Overseer may continue to be a useful tool for farm planning, including nitrogen management, and for preparing farm environment plans, which are mandatory under the Essential Freshwater package.
“The government response to the Science Advisory Panel’s review sets out multiple options for regulatory support, including a completely new nutrient loss model. We consider that several new, fit-for-purpose predictive tools may be needed, in lieu of a single monolithic model. New tools should:
- take advantage of the rapid growth in environmental and agricultural data generation,
- guide the collection of additional data to overcome known deficiencies,
- utilise recent advances in modelling, data science and technology, such as continuous water quality monitoring, and
- ensure that prediction uncertainty is quantified and made transparent to all users.”
Conflict of interest statement: “NIWA scientists contributed to the 2018 PCE review of Overseer. NIWA scientists have developed wetland and riparian modules for Overseer; these modules were not included in the Science Advisory Panel review.”
Anthony Pleasants, Adjunct Professor in Agricultural Analytics, Massey University; Dr Stephen Taylor, Senior Lecturer in Applied Mathematics, University of Auckland; and Dr Graeme Wake, Adjunct Professor of Industrial Mathematics, Auckland University of Technology:
“The number of dairy cattle in Aotearoa New Zealand has doubled in the last two decades to over six million, thereby giving rise to increased farm intensification and capacity. This drove the need for higher grass yield produced by more and sometimes different composition of fertilisers to be used. Both the fertiliser run-off and the nitrogen in urea promotes microbial growth mostly in waterways and can contribute to poor health outcomes and downgrading of our environment. Finding strategies to minimise this impact is a priority for the whole country and should be underpinned by sound science.
“The essential ingredient here is the subtle complexities of interacting temporal effects and circumstantial changes like the weather, run-off, mitigation and dissipation into the catchment (soil and waterways), and the grazing strategy, The accumulation of nitrogen from the animal excrement and the fertiliser not absorbed by the pasture into the catchment areas is the result. Any decision-support model must incorporate all this together, with the inherent statistical variability in both inputs and outcomes. The methods are well known here and widely used internationally. They give bounds and estimates on the grazing system pattern to be used. However, the ingredients in the Overseer FM tool which has been very unwisely but widely used here up to now, is an overly simplistic approach not incorporating temporal “cause and outcome” effects, and system interactions. These are essential. The calculations therein fail to address the key dynamical issues central to the problem. Within Overseer FM there have been even more elementary fundamental computational errors reported.
“In Overseer the key science is not woven together as proper science would dictate. The report now released, suggests that the development of Overseer has, alarmingly, not received input or peer evaluation from any experienced independent system modellers for which New Zealand is quite well-endowed. Some of these scientists even offered to help as the idea was first mooted in the 1990s.
“Rules and procedures should be determined by accepted wisdom and be transparent to those making the rules and to those who must carry them out. Here the science system has grossly failed for Aotearoa New Zealand. This overdue investigation of Overseer is a most welcome step in addressing this unfortunate blot on our scientific practice. Action should follow to bring New Zealand up to better ecological standards, using the collective skills we have. Much better and very different methodologies are certainly needed in our opinion.”
Conflict of interest statement: “All three of us are currently working together in a wider interdisciplinary team from 2020 on a partially externally funded major catchment modelling model development that captures all the essential features.”
Source: Science Media Centre