Effectiveness of climate information services: an evaluation of the accuracy and socio-economic benefits for smallholder farmers in Niger and Mali

Climate information services are foundational means of building the farmer’s resilience. However, studies are scarce about the accuracy of climate information services in dryland regions such as the West Africa Sahel, like in Mali and Niger. Thus, this study examined the accuracy of climate forecasts and their socio-economic benefits in these two countries. For rainfall forecasts and alerts, we collected the 2022 data from the ‘SMS Sandji’ platform in Mali (Nara) and the national meteorological agency alert database in Niger (Zinder). The socio-economic benefits of climate information were determined using a sample of 900 individuals in Niger and 227 in Mali. The results indicate that both seasonal and daily climate forecasts have high to moderate accuracy from 0.7 to 0.58 for CSI and 0.11 to 0.43 for BS index in Niger, and 0.94 to 0.91 for CSI, and 0.06 to 0.25 for BS in Mali. The results of field survey show that, in general, 87 to 100% of the respondents in Niger and 100% in Mali received the seasonal forecasts. ANOVA also reveals with high significance (p value = 0.0001) that the utilization of climate information plays a crucial role in improving farmers’ average financial incomes with FCFA 24,943 per hectare at season onset to FCFA 15,355 per hectare during the cropping season, and FCFA 6204 per hectare at the end of the season, and time-saving of 36 h per hectare to 8 h per hectare, depending on the period when the information was used. Globally, this work underscores the importance of climate information services and highlights their positive socio-economic impacts to the livelihood of farmers.

1 Introduction

In recent years, farmers in Mali and Niger have commenced using climate information systems to understand weather conditions, make informed decisions, and adjust their farming practices accordingly (Traoré et al., 2018; Bejamin et al., 2020). The ability to predict climate fluctuations in days to months ahead could make a real difference in Africans’ adaptation strategies to climate change and be a first step toward increasing productivity and minimizing the risk of food crises (Hassane et al., 2017). Climate variability and change pose significant challenges for agricultural activities, particularly in Sub-Saharan Africa (Traore et al., 2013). Mali and Niger, two landlocked countries in West Africa, heavily rely on rain-fed agriculture as a major source of income and food security for their populations. Yet, unpredictable rainfall patterns and extreme weather events such as droughts and floods frequently disrupt agricultural production, exacerbating food insecurity and poverty in these regions (Lipper et al., 2014; Traore et al., 2021; Traore et al., 2024).

To adapt to increasing climate impacts in West Africa, farmers have used indigenous and modern adaptation strategies. Examples of these techniques include soil and water conservation practices (CIAT et al., 2020), improved varieties (Sawadogo et al., 2017), and most recently, the dissemination of climate information service’s early warning system (Sanoussi et al., 2015; Roudier et al., 2016; Traoré et al., 2018). The upscaling of weather forecasting systems for risk management in agricultural practices sounds relevant as it provides timely weather information, helping producers to make appropriate decisions (Roudier et al., 2012; Sanoussi et al., 2015; Seydou et al., 2023). Many studies have demonstrated farmers’ interest in Africa in seasonal and day-to-day alerts of agro-climatic information for better environmental management to improve crop productivity. However, sharing information with farmers poses many challenges, especially in rural areas of Mali and Niger. Climate information such as rainfall forecasts are generally highly technical and scientific and farmers are not keen to use them. Furthermore, climate information sharing systems in Niger and Mali are often top-down (Roudier et al., 2016). Seasonal climate forecasts have no intrinsic value because they arise from improvement decisions and do not constitute direct climate solutions. There is a need along the agriculture value chain to expand significant amounts of useful and reliable information to which farmers do not have access.

As climate information is critical in farming and the making of household-level decisions, a seasonal forecast platform called PRESAO (Seasonal Forecast in West Africa) was established in 1998 by regional research institutions (ACMAD, AGRHYMET, NBA and ICRISAT). This process, known as PRESASS since 2012, brings together experts and all stakeholders involved in climate issues in West Africa to deliver seasonal forecasts. The main forecasted parameters are: annual rainfall during the season, season onset/end, and duration; and dry spell duration. It has the potential to translate forecasts of climate anomalies into predicted impacts on production and economic incomes (Moussa et al., 2020; Seydou et al., 2023).

The reliability of rainfall alerts and seasonal forecast remain a matter of concern. In most cases, ex-ante methods are the only way to estimate the benefits of using agro-climatic information in agriculture (Roudier et al., 2016; Seydou et al., 2023) and they are the only methods of assessing the potential benefits of support interventions. Despite the importance of climate information systems in Mali and Niger, there is a notable research gap regarding their effectiveness in improving resilience and adaptation to climate change in these countries. While some studies have assessed the technical aspects of these systems, there is limited evidence on their impact at the community and household levels. This paper aims to examine the effectiveness of climate information systems in Mali and Niger, with a focus on their contribution to adaptive capacity and productivity improvement. It also reviews the reliability of rainfall alerts in Mali and Niger, with a focus on understanding the accuracy of predictions, the effectiveness of dissemination mechanisms, and the subsequent impact on farmers’ decision-making processes. By evaluating the strengths and weaknesses of existing rainfall alert systems, this research seeks to address knowledge gaps and contribute to the improvement of these systems for achieving enhanced agricultural resilience, better risk management and productivity.

1.1 Theoretical framework

In Mali and Niger, farmers receive climate information through electronic platforms such as mobile phone applications or websites that provide real-time weather updates and forecasts (Seydou et al., 2023). They can also receive climate information through radio broadcasts, community meetings and workshops, extension services provided by agricultural agencies or NGOs, and through the use of local weather stations and equipment installed on farms (Ouédraogo et al., 2018). Additionally, collaborating with local meteorological agencies and research institutions can provide smallholders with access to more accurate and detailed climate information. As example in Niger, the National Agency of Meteorology works closely with local farmers and provides them with climate data and forecasts tailored to their specific regions via WhatsApp groups and local radio stations. In Nara also, farmers can access climate information via Orange mobile phone network operator service named Sandji. Farmers can request information in SMS or voice call form depending on their preference.

Farmers who have access to timely and reliable climate information are more likely to adopt climate-resilient agricultural practices (Zougmoré et al., 2016). This can include adopting drought-resistant crop varieties, adjusting planting and harvesting schedules based on weather forecasts, implementing efficient irrigation practices, and developing contingency plans for extreme weather events (Diouf et al., 2020). To promote climate-resilient agriculture and minimize the risks imposed by climate change, it is crucial to provide smallholders farmers with access to reliable and accurate climate information. This will enable them to make informed decisions about their farming practices, mitigate the adverse effects of climate change, and adapt to changing climatic conditions.

The clarification of the following concepts will help us to better understand the context in which we conducted this research work and the implications of our findings. The concepts that need clarification in the context of this research work are:

✓ Climate Information:

This refers to data and knowledge about past, present, and future climatic conditions and trends. Such information includes weather forecasts, climate projections, historical climate data, and information on climate variability and extreme events (Hansen et al., 2019). Weather forecasts and climate projections are crucial components of climate information in agriculture. They provide valuable insights into future climatic conditions, allowing farmers and policymakers to make informed decisions regarding agricultural practices and management strategies (FAO, 2019). Hence, climate information plays a critical role in climate risk reduction and adaptation in agricultural systems.

✓ Climate Information system:

Refers to a system that integrates climate and weather data, along with agronomic information, to provide farmers with timely and relevant information for decision-making in relation to climate risk reduction and adaptation in agricultural systems (WMO, 2014).

✓ Climate Risk Reduction:

This refers to the implementation of measures and strategies aimed at minimizing the adverse impacts of climate change on agricultural systems (FAO, 2019). These measures can include the use of climate information to inform decision-making processes, such as adjusting planting dates, selecting drought-resistant crop varieties, and implementing irrigation.

✓ Smallholder farmers:

According to FAO, smallholder farmers are small-scale farmers, pastoralists, forest keepers, fishers who manage areas varying from less than one hectare to 10 hectares. Smallholders are characterized by family-focused motives such as favoring the stability of the farm household system, using mainly family labor for production and using part of the produce for family consumption.

✓ Effectiveness of climate information:

Is defined as the extent to which climate information contributes to reducing climate risks and enhancing the adaptive capacity of agricultural systems (WMO, 2014). It depends on several factors, including the accuracy and reliability of the information, timely delivery to end-users, accessibility of the information, and the capacity of farmers to understand and utilize the information effectively. Farmers rely on climate information for making informed decisions regarding agricultural practices and management strategies.

✓ Accuracy of climate information:

Refers to the degree to which the information reflects the actual climatic conditions and predictions source and the level of agreement between observed and predicted weather patterns (Amadi and Chigbu, 2014). Furthermore, the reliability of climate information refers to its consistency and trustworthiness over time.

2 Materials and methods

2.1 Material

Quantitative data were collected from 1,127 farmers (900 in Niger and 227 in Mali). Data were collected in four (4) communes (Gueneibe, Koronga, Nara and Ouagadou) in Nara region (Mali), and eight (8) communes (Diffa, Chetimari, Dungass, Magaria, Wacha, S Broum, Gazaoua and Koona) in Diffa, Zinder and Maradi regions (Niger). We used the following tools for information collection and analyses:

• An individual questionnaire, including questions on identification and description, types of agro-climate services received, actions taken to mitigate the anticipated responses to climatic hazard and socio-economic benefits obtained.

• Open Data Kit collect (ODK) software, used to develop the questionnaire, synthesize responses, process and analyze field survey data, and extract questionnaire results.

2.2 Methods

2.2.1 Site

The study is conducted in Niger and Mali where climate typology varies from Sudanese to Sahelian. The agrarian system of these countries is still slightly rudimentary and very sensitive to climate variability and risks. This exacerbates the prevailing fragile food systems and increases risk of food insecurity.

In Mali, this study is conducted in Nara, in the arid western Sahel region where climate is typical of the Sahelian zone. Annual rainfall in that region varies between 300 to 600 mm and temperature peaks reach 46°C during the hot season (Figure 1). The rainy season extends from May to October and the seasonal average temperature is 31°C. The most common farming systems in the region are extensive mixed agro-sylvo-pastoral systems.

Figure 1

Figure 1. Elevation map of the study area.

In Niger, this study is conducted in the Diffa, Maradi and Zinder regions, one of the major agricultural zones of the Central-Eastern Niger. These regions are located in a typical Sahelian agricultural zone, with an average 400–600 mm annual rainfall (Ado et al., 2018). Rainfed agriculture occupies more than 95% of the population (Moussa et al., 2020). Crops are generally grown on dune lands and lowlands using manual or animal-drawn hoes. The main crops are millet, sorghum and peanuts, which are mainly used for self-consumption by a rapidly growing population.

The choice of these locations as study regions can be justified by the fact that the climatic and demographic dynamics underway place them at the heart of major agricultural changes, between adaptation, improving food security and sustainable agriculture. Also, several actors have implemented a wide network of climate information dissemination. This allowed us to endorse them and measure the impact of these interventions.

2.2.2 Meteorological data observation and analyses

• Source of data

To assess the accuracy of rainfall alerts, we subscribed farmers to the Sandji service provided by Orange Mali¹. Sandji is a mobile phone platform for the dissemination of climate data that enables farmers to request, for example, for rainfall forecasts in good time in order to anticipate actions to be taken in the face of uncertainty. In Niger, we considered the daily alerts shared by the national meteorological agency via the WhatsApp (social media) platform, and rural community radio broadcasts. These alerts were compared with real rainfall measured with rain gauges installed in the villages.

To assess the accuracy of seasonal forecasts, we compared the 12-year Seasonal Forecasts of the Agro-hydro-climatic characteristics of the main rainy season for the Sudanian and Sahelian zones (PRESASS) with the daily meteorological data obtained from the National Meteorological Service of Niger from 2011 to 2022.

• Calculation of The Brier Score

The Brier Score was calculated for each forecast and alert by comparing the forecasted probability or alert status against the observed outcome. The formula for calculating the Brier Score (Rouseau, 2001) is as follows:

$\text{Brier}\text{Score}=\sum {\left(\mathrm{P}-\mathrm{O}\right)}^2$

Where P represents the forecasted probability or alert status (0 or 1) and O represents the observed outcome (0 or 1). A lower Brier Score indicates higher forecast accuracy.

• Calculation of Critical Success Index (CSI):

To compute the Critical Success Index (CSI) (Qi et al., 2016) for climate alert forecast accuracy, we compared the forecasted events with the observed events. The CSI is a measure that quantifies the accuracy of detecting correctly forecasted events. The formula for calculating the CSI (Rouseau, 2001; Qi et al., 2016) is as follows:

$\mathrm{C}\mathrm{S}\mathrm{I}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{F}\mathrm{P}}$

- True Positive (TP): The number of correctly forecasted events.

- False Positive (FP): The number of forecasted events that did not occur.

- False Negative (FN): The number of observed events that were not forecasted.

The CSI ranges from 0 to 1, with 1 indicating perfect accuracy.

• Calculation of categorical accuracy

Categorical accuracy is a metric used to evaluate rainfall alerts. It measures the proportion of correctly classified instances out of the total instances. To calculate categorical accuracy, we used the following formula (Gordon, 1982):

$\text{Categorical}\text{Accuracy}=\frac{\text{Number}\text{of}\text{Correct}\text{Predictions}}{\text{Total}\text{Number}\text{of}\text{Predictions}}\times 100$

2.2.3 Socioeconomic data collection and analyses

• Sampling method

We collected data from a sample of farmers benefiting from a climate information system. Farmers subscribe to electronic platforms (Sandji and Niger-Meteo WhatsApp groups) or local radio broadcast stations to receive climate information (rainfall forecasts, seasonal forecasts, floods, etc.). We calculated the sample size using the following formula (Cochran, 1963):

$\text{Sample}\text{size}=\frac{{\mathrm{Z}}^{2}P\left(1-P\right)}{E^2}$

Z = confidence level; P = estimated proportion; E = margin of error.

A confidence level of 97, 3% margin of error, and 70% of population proportion were estimated.

$\text{Sample}\text{size}=\frac{{1.88}^2\ast 0.7\ast \left(1-0.7\right)}{{0.03}^2}$

$\text{Sample}\text{size}=824.6933$

• Data Collection and Analysis

We collected Data through individual interviews. The collected data is analyzed using appropriate statistical techniques, including descriptive statistics and inferential analysis such as analysis of variance (ANOVA). ANOVA is used to assess the significance of differences in the perceived socio-economic gains and time savings among different regions of Niger.

We used the following formula for ANOVA (Yang et al., 2014):

$S^2=1/n\sum x{i}^2-x^2$

whit i = 1 to n.

3 Results

3.1 Accuracy of rainfall forecast system in Mali and Niger

3.1.1 Evaluation of the accuracy of Sandji rainfall forecast service in Nara (Mali)

• Proportion of successful and unsuccessful “No rain” alert

Table 1 illustrates the percentage of successful and unsuccessful “no rain” alert in multiple villages of Nara region. The results show that the “no rain” forecasts were mostly accurate, with success rates ranging from 84 to 100%. The month of September was the most successful month for “no rain” alert, with a 100% success record rate in some localities across the region (Bezak; Breguenaré; Goumo; Guenebé; Kabida Bambara; Kabida Soniké; Keibane Maure; Keibane Soniké).

Table 1

Table 1. Proportion of successful and unsuccessful “No rain” alert in Nara from August to November 2022.

In unsuccessful alerts, recorded percentages are found much lower than in successful alerts, with values ranging from 0 to 15%. The highest failure rate was observed in Dyagaba and Nara villages in August and October (Table 1).

• Proportion of successful and unsuccessful “rain alert”

Table 2 below shows the proportion of successful and unsuccessful “rain forecasts” in the Nara region of Mali. The result shows that for “rain alert” forecasts, the success rate is much higher than the failure rate, excluding the months of September and November when the two rates were 50–50 in Soutourabougou and Moussawelli.

Table 2

Table 2. Proportion of successful and unsuccessful “rain alert” in Nara from August to November 2022.

The unsuccessful alert rates are mostly lower than successes with percentages varying from relatively low to moderate (Table 2)

• Sandji forecast service accuracy using BS an CSI

Figure 2 shows the accuracy of the Sandji alert system in Nara (Mali) using Critical Success Index (CSI) and Brier Score (BS) during the year 2022. The high proportion of forecasted and achieved rainfall activities lead to a Critical Success Index too close to 1 (0.91 in August and 0.94 in September, October and November). The same trend is observed with the Brier Score, i.e., very close to 0 (0.05 in August and September and 0.25 and 0.06 in October and November, respectively).

Figure 2

Figure 2. Evaluation of the accuracy of the Sandji alert system in Nara (Mali) using Critical Success Index (CSI) and Brier Score (BS) during the year 2022 for a better climate risk reduction (TP, True Positive; FP, False Positive; FN, False Negative).

3.1.2 Evaluation of the accuracy of meteorological alert in Magaria, Niger

• Proportion of successful and unsuccessful “No rain alert”

Table 3 shows the percentage of successful and unsuccessful “No rain alert” from the national meteorological agency in Magaria department of Niger during the months of May–September. In May, the predictions were 100% accurate. They were more than 77% correct in June, with failure rates ranging from 0 to 22.2%. The same trend continued through September, although the highest failure rates of up to 45% in localities including Agoual gamji, Jan mage, Katirge, were recorded in September.

• Proportion of successful and unsuccessful “rain alert”

Table 3

Table 3. Accuracy of “no rain alert” in the department of Magaria, Zinder region, Niger, from May to September 2022.

Table 4 presents the success and failure rates of “rain alert” by village during the months of June–September 2022. No “rain alert” was issued in May. In June, the rate of unsuccessful alert was much higher (75–100%) than the number of successful cases (0–25%). In July, the failure rate still prevailed, ranging from 55.6 to 100%, compared to success rate of only 11.1 to 44%. Comparatively, in August, the success rates, 50 to 87.5%, were higher than the failure rates, i.e., 12.5 to 50%. Finally, in September, the success versus failure rates were almost the same.

• Meto alert system accuracy using BS an CSI in Magaria (Niger)

Table 4

Table 4. Accuracy of national meteorological forecast for “rain alert” in Magaria department (Zinder region in Niger) from May to September 2022.

Figure 3 illustrates the evaluation of rainfall forecasts using the Critical Success Index and Brier Score. In May, the Critical Success Index of 1 and a Brier Score of 0 imply that the rainfall forecasts predicted the occurrence of rainfall accurately. This indicates the effectiveness of the forecasting system in predicting rain events. The least accurate forecasts were observed in August with a CSI of 0.69 and a BS of 0.43.

Figure 3

Figure 3. Evaluation of the accuracy of the meteorological alert system in Magaria department (Niger) using Critical Success Index (CSI) and Brier Score (BS) during the year 2022 a better climate risk reduction. TP, True Positive; FP, False Positive; FN, False Negative.

3.2 Accuracy of the seasonal forecast in Maradi, Zinder and Diffa, Niger

• Proportion of successful and wrong seasonal forecasts

Figure 4 below shows the proportion of correct and wrong forecasts for rainy season onset, dry spells at the beginning of the season, quantity of annual rainfall, occurrence of dry spells at the end of the season, and date of the end of the season.

Figure 4

Figure 4. Accuracy of seasonal forecasts for five crucial parameters (rainy season onset, dry spell at the beginning of the season, quantity of annual rainfall, dry spell at the end of the season, and date of end of the season) from 2011 to 2022 in Maradi, Zinder, and Diffa (Niger).

The forecasts of the rainy season onset were mostly accurate, with success rates ranging from 58 to 83%. However, Diffa recorded the highest failure rate (42%), almost half of the total forecast. For the date of the end of the season, the results also show a very high success rate, i.e., over 80% in all regions. Regarding forecasts of dry spells at season onset, the same trend continues, with fairly high success rates in Diffa (90%), Maradi (70%) and Zinder (80%). Comparatively, forecasts of dry spells at the end of the season indicate a slightly high failure rate (40%) in Diffa and Maradi. Finally, for total cumulative rainfall, the forecasts were correct except in Diffa where the failure rate was 42%.

• Evaluation of the accuracy of seasonal forecast parameters

Figure 5 shows that all forecasts are accurate with a > 60% categorical accuracy, and a BS varying from 0.2 to 0.4 in Maradi, Zinder and Diffa. For dry spells, the most accurate forecasts were observed at the biggening of the season and the end of the rainy season, with a BS of 0.1 and a 90% categorical accuracy in Diffa region. The least accurate forecast was observed for total rainfall in Zinder and rainy season onset in Diffa with a BS of 0.4 and a categorical accuracy of 58%.

Figure 5

Figure 5. Accuracy of seasonal forecasts for five crucial parameters (rainy season onset, dry spell at the beginning of the season, quantity of annual rainfall, dry spell at the end of the season, and date of end of the season) using categorical accuracy and Brier Score from 2011 to 2022 in Maradi, Zinder, and Diffa, Niger.

3.3 The use of climate information and socioeconomic benefit for a better climate risk reduction

3.3.1 Type of climate information delivered and channel of reception

Figure 6 presents the types of climate information received by farmers in the 2022 rainy season, and the main channels of dissemination ranked by percentage.

Figure 6

Figure 6. Climate information received in 2022 rainy season and channel of reception in the study area (A = climate information received in Mali; B = Channel of reception in Mali; C = climate information received in Niger; D = Channel of reception in Niger).

The result indicates that in Mali, producers received much more rainfall forecasts (Figure 5A). Seasonal forecasts constitute the second most common climate information received by farmers in the surveyed area. The least frequently received climate information was the dry spells forecast. Comparatively, in Niger, seasonal forecasts represent a significant portion of the information received (Figure 5C), followed by flood prediction (45% in Maradi, 61% in Diffa and 55% in Zinder) and rain forecasts (49% in Diffa, 60% in Maradi and 50% in Zinder). The least-received forecasts are: dry spell, temperature, and wind.

In terms of channels of reception, in Mali, the surveyed farmers affirmed local radio broadcasts and SMS alerts to be the most used channels for climate information delivery (Figure 5B). However, a small portion of the group (12% at Gueneibe, 1% at Nara and 3% Ouagadou) affirmed that the farmers’ organization has been another channel for climate information dissemination.

The same communication channels are used in Niger. Farmers’ organizations emerge as the primary source, providing reliable climate information to a significant group of beneficiaries (Figure 5D). Additionally, farmers rely on information shared by local radio station (88% in Diffa), emphasizing the significance of large audience of this mean of communication in rural areas. Neighbors and parents (from 18 to 32%) play a crucial role too, acting as a reliable channel for dissemination of climate information. Likewise, the role of extension services (11 to 14%) and mobile phone calls (12 to 17%) in delivering climate information shall not be neglected (Figure 5D).

3.3.2 Concrete actions taken by farmers and deriving socio-economic benefits

Figure 7 illustrates the actions taken by farmers after receiving climate information and some socio-economic implications.

Figure 7

Figure 7. Modification of farming activities and the socio-economic benefits of using climate information for a better climate risk reduction (A = modification of farming activities at the biggening of the farming season; B = modification of farming activities during the farming season; C = Modification of farming activities at the end of the farming season; D = socio-economic benefits of using climate information at the biggening of the farming season; E = socio-economic benefits of using climate information during the farming season; F = socio-economic benefits of using climate information at the end of the farming season).

At the beginning of the rainy season, these actions encompass selecting the right variety, determining the optimal sowing period, manure application, and plowing periods (Figure 6A). Some producers made informed decisions with regard to choice of production site, land preparation, and the site size.

From the actions taken, the analysis of variance discloses a significant difference (p value = 0.0001) between regions in terms of economic gain and time saving. Farmers in Diffa experienced a substantial gain of approximately FCFA 25,000, followed by Zinder farmers (≈ FCFA 17,000). Moreover, farmers in Maradi also registered a significant gain, though slightly lower, i.e., ≈ FCFA 15,000 (Figure 5D). In addition to economic gains, the results reveal substantial time savings. Farmers in different regions could save invaluable hours: 36 h in Zinder, 35 h in Maradi, and 25 h in Diffa (Figure 6D).

The actual actions taken by farmers based on climate information received during the rainy season include: modification of sowing period, period of fertilizer application: manure, NPK, urea (Figure 6B). Other actions such as change of period of pesticide application, manual weeding, earthing up/weeding are taken by a few number of surveyed farmers.

During the rainy season also, the analysis of variance of the survey results shows very significant differences (p value = 0.0001) between regions, both in terms of economic benefits and reduction of wasted man/h time. Diffa recorded the highest gain of ≈ CFAF 15,000, followed by Zinder (CFAF 12,000), and Maradi (CFAF 11,000). In terms of reduction of man/h time loss, the highest number of lost hours recovered by farmers was recorded in Maradi (22 h), followed by Zinder (13 h). In Diffa, the man/h time saving was lower (11 h), but equally significant per hectare (Figure 6E).

At the end of the rainy season, modifications primarily focus on period of harvesting (>90%) and grain processing (Figure 6C). Applying the ANOVA reveals a high significant difference between region (p value = 0.001) in terms of economic gain and time saving. Farmers in Maradi experienced a substantial gain of ≈ CFA 62,000, reflecting the positive influence of informed decision-making on their economic incomes. Following Maradi is Zinder, where the economic gain was also remarkable, i.e., approximately CFA 54,000. Additionally, farmers in Diffa also experienced a substantial gain, albeit slightly lower, at ≈ CFA 42,000 (Figure 6F). Furthermore, farmers in different regions saved invaluable hours: roughly 9 h at Diffa and Maradi, and 7 h at Zinder.

4 Discussion

The discussion covered the outcomes achieved from the evaluation of climate forecast tools.

The general theme presents the commendable accuracy of these tools that provide farmers with diverse climate information, thus enabling them to make informed decisions, resulting in significant financial and time gains and preparedness to climate risk. The positive correlation between the effectiveness of climate services and farmers’ agricultural decision-making highlights the invaluable role these tools play in improving productivity and resource management. Globally, the results of this study demonstrate that climate information services play a crucial role in supporting agricultural decision-making in Mali and Niger. Farmers perceive these services as highly reliable and important for making informed decisions. The good accuracy of seasonal forecasts and rainfall alerts further reinforces the effectiveness of these services. The use of climate information services not only helps farmers increase their income but also saves their time and efforts, ultimately contributing to improved agricultural productivity, risk management and sustainability.

4.1 Accuracy of rainfall alerts

In Mali, our result highlights the high accuracy of Sandji alert system, with a true detection rate much higher than the failure rate. This underscores the effectiveness of the Sandji alert system in detecting and predicting rainfall. This result proves the alert system’s ability to provide farmers in the Nara region with the necessary information to make the right decisions and cope with climate uncertainty and risk. An accurate rainfall forecast plays a crucial role in climate adaption and mitigation strategies, especially for small-scale farmers who heavily rely on rainfed agriculture (Roudier et al., 2016; Ouédraogo et al., 2018). Another major advantage of the Sandji system is the permanent availability of climatic information. This result confirms Ouédraogo et al. (2018) study on the willingness of farmers to pay for climate information, due to its importance for them. To enhance the uptake of climate-smart agricultural practices and improve farmers’ preparedness for climate-related risks, accurate weather forecasts are of utmost importance. By integrating accessible and digital channels of climate information delivery, we can bridge the gap between meteorological departments and small-scale farmers. It’s important to note that the result shows a more accurate prediction for “no rain” alerts than for “rain” alerts. Hence a need to further improve the model.

In Niger, the findings of this study highlight the accuracy and effectiveness of climate information services in supporting agricultural decision-making in Niger. The results indicate that climate information services are highly reliable and important for making informed decisions. This result corroborates with Amadi and Chigbu (2014) on the need for accurate and timely weather and climate services. Additionally, the assessment of the accuracy of seasonal forecasts and rainfall alerts using both the Brier Score and the Critical Success Index indicates good accuracy of these services. Globally, the result is undoubtedly encouraging as it demonstrates the positive impact of advanced forecasting techniques and the dedication put into developing accurate alert systems. Such information can greatly assist decision-making processes in sectors such as agriculture, ultimately benefiting the communities in the villages of Magaria (Niger) as demonstrated by Nkiaka et al. (2019) in their research conducted in Nigeria. In deduction, this result highlights the success of rainfall alert systems in various villages in the study area. The significant percentage of successful alerts and very good indexes ensure the reliability of the system’s predictions and serves as an invaluable tool for the local communities and organizations operating in these regions. As illustrated by Awolala et al. (2022) in their results, accurate climate information makes a major contribution to risk preparedness. However, the results also expose a notable trend in which forecasts show greater accuracy in predicting “no rain” as compared to “rain” alerts. This discrepancy represents a potential challenge in the accuracy of weather forecast during the wettest period of the farming season. It is important that farmers and stakeholders are made aware of this challenge, as well as the need for enhancement. Future improvements in modeling and data assimilation techniques could improve the reliability of rain alerts, thus narrowing the current gap in forecast accuracy between “rain” and “no rain” alerts.

It is important to notice the transferability of the framework presented in this paper to other African countries. The potential for transferability of the framework highlights its relevance and applicability in addressing climate risks in the region. Additionally, the framework demonstrated significant accuracy and theories that have been widely demonstrated and applied in coping and adaptation strategies. Nevertheless, the perishable character of climate information requires that the framework can be implemented in diverse zones if only climate forecasts are available in the zone and network coverage is enough to provide accurate data to end beneficiaries. Therefore, considering the unique needs and challenges of each zone is crucial to ensure the successful transferability and implementation of the model.

4.2 Accuracy of the seasonal forecast

The evaluation of the accuracy of seasonal forecasts and rainfall alerts using metrics such as the Brier Score and Critical Success Index further strengthens the notion of effectiveness of climate information services. The good performance of these forecasts and alerts implies that they provide reliable and timely information to farmers, which aids in planning agricultural activities (Hansen et al., 2011). This plays a crucial role on another important aspect of the study which is farmers’ perception of the reliability and usefulness of climate information services. The high level of trust in these services suggests that farmers welcome them as dependable sources of information allowing them to confidently make decisions based on received climate information, which lead to improved agricultural practices also refer to findings by Djido et al. (2021) and Seydou et al. (2023). It is evident that seasonal forecasts provide broader possibilities for modifying farming activities, allowing farmers to plan and adjust practices over an extended period of time. This is in contrast to alerts, which suggest more immediate modifications, often on day-to-day bases. The flexibility of seasonal forecasts enables farmers to make strategic decisions, considering long-term trends and potential variations in weather patterns. Comparatively, alerts play a crucial role in responding to short-term changes, offering farmers the ability to swiftly adapt to sudden weather shifts. Combining these approaches can create a comprehensive strategy, blending long-term planning with real-time adjustments for optimal farming outcomes.

4.3 Type of climate information delivered and channel of reception

Climate services are created in multiple installments depending on the producer’s preferences and the optimal time period for the message to be useful, because climate information is a highly perishable input. One of the essential components of climate information is the distribution channels, as they form the primary means of conveying messages from the researcher or expert to the ultimate beneficiary, the farmer (Vincent et al., 2020).

The type of climate information that producers receive the most in Niger is seasonal forecasts, followed by flood and drought forecasts. These informations are relayed through channels such as farmers’ organizations and radio stations. The two channels play key roles in disseminating the CIS. This result differs from the finding by Seydou et al. (2023), who argue that producers mandated by farmer organizations to attend seasonal forecast forum do not share back their knowledge with the members who did not attend the meeting. In Mali, producers of the surveyed sample received more rainfall forecasts, then seasonal forecasts. In Mali and Niger, rural radio stations play an important role in disseminating climate information. However, in Mali more than in Niger, farmers also favor SMS alerts which are heavily used (Traoré et al., 2018). These results confirm many studies in the subregion showing that over 90% of producers who have acted upon climate information received via mobile telephone messaging to improve farming practices, have actually achieved an increase of their production (Traoré et al., 2018; Sidibé et al., 2021; Traore et al., 2021).

In summary, farmers receive more seasonal forecasts in Niger and more rain forecasts in Mali. In one context, the prevalence of rain forecasts in Mali suggests a focus on immediate weather events, potentially driven by the agrarian significance of rainfall. This could be attributed to efficient channels such as farmers’ organizations, SMS services, local radios and extension services, serving as pivotal conduits for disseminating crucial meteorological information. Meanwhile, in Niger, the emphasis on seasonal forecasts implies a broader consideration of long-term planning, with distinct channels playing a prominent role in ensuring that farmers receive this specific type of information. Understanding these changes in information flows is essential for optimizing communication strategies and improving farmer preparedness and decision-making attitude in diverse agricultural activities.

4.4 Concrete actions taken by farmers due to climate information received and its socio-economic benefits

The findings of this study have revealed that farmers have successfully converted climate information into actionable measures for their farming practices. Specifically, the analysis focused on the utilization of seasonal forecasts and their impact on different farming periods, as well as the incorporation of rain forecasts, dry spells, temperature fluctuations, and other day-to-day modifications. The conversion of climate information into concrete actions by farmers is of critical importance, considering the inherent uncertainties and fluctuations associated with weather patterns. Our study highlights the adaptive capacity of farmers to interpret and apply climate forecasts effectively, leading to more productive and resilient agricultural systems.

One key aspect of this study is the use of seasonal forecasts, which allows farmers to modify their farming practices across the entire cropping season. By incorporating climate predictions, farmers can anticipate potential challenges and adjust their strategies accordingly. This proactive approach aids in modifying land preparation and planting periods, the site size, the choice of the variety for planting, and the overall management of various agricultural activities.

Furthermore, the results show that rainfall forecasts play an important role in guiding farmers’ decision-making process. Information about expected rainfall allows farmers to plan planting times, period for applying manure, chemical fertilizers, and pesticides, and weeding. This level of preparation significantly reduces the risks associated with erratic rainfall and improves crop yields.

Another significant finding of this study is the positive impact of climate information services on the economic well-being of farmers. The analysis of variance in the survey results indicates significant regional differences in both economic benefits and reduction of time loss. In the beginning of the farming season, Farmers in Diffa experienced a substantial gain of approximately FCFA 25,000, followed by Zinder at ≈ FCFA 17,000. Furthermore, farmers in Maradi also logged a significant gain, although slightly lower at ≈ FCFA 15,000. During the rainy season, Diffa recorded the largest increase of ≈vFCFA 15,000, followed by Zinder with FCFA 12,000, and Maradi FCFA 11,000. In terms of time loss reduction, the highest number of hours saved by farmers was recorded in Maradi (22 h), followed by Zinder (13 h). In Diffa, the average saved hours was the lowest (11 h) but also significant per hectare. At the end of the season, Diffa stands out with the highest economic gain of approximately FFA 15,000, followed by Zinder at FCFA 12,000 and Maradi at FCFA 11,000. Notably, Maradi experienced the most substantial reduction in time loss with an average saving of 22 h per farmer, while Zinder saved 13 h on average. Although Diffa recorded the smallest time savings (11 h per hectare), the impact remains significant. This result confirms the findings by Roudier et al. (2016) and Seydou et al. (2023), in terms of socio economic benefit of climate services. Definitely, by using these services, farmers are able to make decisions that not only lead to increased crop yields but also contribute in saving their man/h time and efforts. The timely and accurate information provided by these services allow farmers to adjust their agricultural practices and take advantage of favorable weather conditions, thereby increasing their productivity and profitability. Moreover, the ability to save work time and efforts through well-informed decision-making allows farmers to allocate their resources more effectively, leading to improved efficiency.

5 Conclusion

This study provides evidence of the effectiveness of climate information services to support agricultural decision-making in Mali and Niger. The results show that farmers appreciate and have trust in these services that they consider as a reliable source of information. This awareness allows farmers to make informed decisions, leading to better agricultural practices and, furthermore, to implementing better climate adaptive farming systems for climate risk reduction and better adaptation to changing weather patterns.

Assessing the accuracy of seasonal forecasts and precipitation warnings using Brier Scores and Critical Success Indicators further illustrates the effectiveness of climate information services. These services provide reliable and timely information, thereby helping farmers plan agricultural activities more effectively. Accurate seasonal forecasts also enable farmers to make informed decisions across all seasonal farming operations, while timely rainfall warnings help adjust daily decisions for better risk management.

Furthermore, there is a positive impact of climate information services on farmers’ economic incomes. By using these services, farmers can make decisions that increase profits and productivity. The ability to adjust agricultural activities based on reliable information allows for optimal resource allocation and taking advantage of favorable weather conditions. Additionally, the time and effort saved through informed decisions improve the efficiency.

In a general way, seasonal forecasts are relevant for yearly planting activities planning whereas rainfall alerts are day-to-day decision-making tools. Incorporating seasonal forecasts and rainfall alerts into farming activities planning can help optimize decision-making and increase the chances of climate risk management and successful crop production. Therefore, the study recommends the use of climate information services for national climate policy, projects and programs for climate-related risk reduction, and building sustainable socio-economic resilience of the farmers in Mali and Niger.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the [patients/ participants OR patients/participants legal guardian/next of kin] was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

IB: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Writing – original draft, Writing – review & editing. BT: Conceptualization, Methodology, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing. AS: Conceptualization, Methodology, Supervision, Validation, Writing – review & editing. MS: Conceptualization, Methodology, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was part of West Africa Science Service Center for Climate Change and Adapted Land use (WASCAL) scholarship program funded by German federal ministry of research and higher education (BMBF). APC charge was supported by Accelerating Impacts of CGIAR Climate Research for Africa (AICCRA) project in Mali.

Acknowledgments

We thank Accelerating Impacts of CGIAR Climate Research for Africa (AICCRA) for supporting data collection in Mali. We would like to thank the National Meteorological Agency of Niger (DMN) for their collaboration during data collection in Niger. Special thanks to Sery Ibrahima COULIBALY for data collection in Mali and, Yacouba DIALLO and Kaboro SAMASSE for their support during the PhD work progress monitoring.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^Orange Mali is a mobile phone services provider that covers locations of the country including the study area.

References

Ado, A. M., Leshan, J., Savadogo, P., Kollin, S., John, K., and Pesha, C. (2018). Households’ vulnerability to climate change: insights from a farming community in Aguie district of Niger. J. Environ. Earth Sci. 8, 127–137.

Google Scholar

Amadi, S. O., and Chigbu, T. O. (2014). The need for accurate and timely weather and climate services in a developing economy: Nigeria in perspective. J. Curr. Res. Sci. 2, 680–686.

Google Scholar

Awolala, D. O., Mutemi, J., Adefisan, E., Antwi-agyei, P., and Taylor, A. (2022). Profiling user needs for weather and climate information in fostering drought risk preparedness in Central-Southern Nigeria. Front. Clim. 4:787605. doi: 10.3389/fclim.2022.787605

Crossref Full Text | Google Scholar

Bejamin, S., Lejeune, Q., Menke, I., Maskell, G., Lee, K., Noblet, M., et al. (2020). Current needs for climate services in West Africa: results from two stakeholder surveys. Clim. Ser. 18:100166. doi: 10.1016/j.cliser.2020.100166

Crossref Full Text | Google Scholar

CIAT, ICRISAT and BFS/USAID. (2020). Climate-Smart Agriculture in Niger. CSA Country Profiles for Africa Series. International Center for Tropical Agriculture (CIAT); International Crops Research Institute for the Semi-Arid Tropics (ICRISAT); Bureau for Food Security, United States Agency for International Development (BFS/ USAID). 28.

Google Scholar

Cochran, W. G. (1963). Sampling techniques, third edition, page 75 read with 51. JohnWiley and Sons, Inc, New York, USA

Google Scholar

Diouf, N. S., Ouedraogo, M., Ouedraogo, I., Ablouka, G., and Zougmoré, R. (2020). Using seasonal forecast as an adaptation strategy: gender differential impact on yield and income in Senegal. Atmos. 11:1127. doi: 10.3390/atmos11101127

Crossref Full Text | Google Scholar

Djido, A., Zougmoré, R. B., Houessionon, P., Ouédraogo, M., Ouédraogo, I., and Seynabou Diouf, N. (2021). To what extent do weather and climate information services drive the adoption of climate-smart agriculture practices in Ghana? Climate. Risk Manage. 32:100309. doi: 10.1016/j.crm.2021.100309

Crossref Full Text | Google Scholar

FAO (2019). Handbook on climate information for farming communities – What farmers need and what is available. Licence: CC BY-NC-SA 3.0 IGO. Rome. 184.

Google Scholar

Gordon, N. D. (1982). Evaluating the skill of categorical forecasts. Mon. Weather Rev. 110, 657–661. doi: 10.1175/1520-0493(1982)110<0657:ETSOCF>2.0.CO;2

PubMed Abstract | Crossref Full Text | Google Scholar

Hansen, J. W., Mason, S. J., Sun, L., and Tall, A. (2011). Review of seasonal climate forecasting for agriculture in sub-Saharan Africa. Exp. Agric. 47, 205–240. doi: 10.1017/S0014479710000876

Crossref Full Text | Google Scholar

Hassane, R., Gastineau, B., Guichard, F., Sultan, B., and Vishel, T. (2017). Analyse environnement et changement climatique en milieu rural au Niger. Available at: https://hal.archives-ouvertes.fr/hal-01772530

Google Scholar

Hansen, J. W., Vaughan, C., Kagabo, D. M., Dinku, T., Carr, E. R., Körner, J., et al. (2019). Climate Services Can Support African Farmers’ Context-Specific Adaptation Needs at Scale. Front. Sustain. Food Syst. 3:21. doi: 10.3389/fsufs.2019.00021

Crossref Full Text | Google Scholar

Lipper, L., Thornton, P., Campbell, B. M., Baedeker, T., Braimoh, A., Bwalya, M., et al. (2014). Climate-smart agriculture for food security. Nat. Clim. Chang. 4, 1068–1072. doi: 10.1038/nclimate2437

PubMed Abstract | Crossref Full Text | Google Scholar

Moussa, M. A., Luc, D., Bachir, A., and Ibrahim, M. (2020). Caractérisation des saisons agricoles au Sahel: analyse des données agro-climatiques versus vécu paysan, cas de la Région de Zinder. Niger 17, 83–101.

Google Scholar

Nkiaka, E., Taylor, A., Dougill, A. J., Antwi-Agyei, P., Fournier, N., Bosire, E. N., et al. (2019). Identifying user needs for weather and climate services to enhance resilience to climate shocks in sub-Saharan Africa. Environ. Res. Lett. 14:13. doi: 10.1088/1748-9326/ab4dfe

Crossref Full Text | Google Scholar

Ouédraogo, M., Barry, S., Zougmoré, R. B., Partey, S. T., Somé, L., and Baki, G. (2018). Farmers’ willingness to pay for climate information services: evidence from cowpea and sesame producers in Northern Burkina Faso. Sustainability 10:36. doi: 10.3390/su10030611

Crossref Full Text | Google Scholar

Qi, W., Zhang, C., Fu, G., Sweetapple, C., and Zhou, H. (2016). Evaluation of global fine-resolution precipitation products and their uncertainty quantification in ensemble discharge simulations. Hydrol. Earth Syst. Sci. 20, 903–920. doi: 10.5194/hess-20-903-2016

Crossref Full Text | Google Scholar

Roudier, P., Alhassane, A., Baron, C., Louvet, S., and Sultan, B. (2016). Assessing the benefits of weather and seasonal forecasts to millet growers in Niger. Agric. For. Meteorol. 223, 168–180. doi: 10.1016/j.agrformet.2016.04.010

Crossref Full Text | Google Scholar

Roudier, P., Sultan, B., Quirion, P., Baron, C., Alhassane, A., Traoré, S. B., et al. (2012). An ex-ante evaluation of the use of seasonal climate forecasts for millet growers in SW Niger. Int. J. Climatol. 32, 759–771. doi: 10.1002/joc.2308

Crossref Full Text | Google Scholar

Rouseau, D. (2001). Un critère général d’évaluation des prévisions déterministes et probabilistes - Application aux prévisions locales diffusées sur Internet. La Météorologie 8:48. doi: 10.4267/2042/36194

Crossref Full Text | Google Scholar

Sanoussi, A., Mohamed, L., Seyni, S., and David, A. G. (2015). Adapting to climate variability and change in smallholder farming communities: a case study from Burkina Faso, Chad and Niger. J. Agric. Extens. Rural Dev. 7, 16–27. doi: 10.5897/JAERD14.0595

Crossref Full Text | Google Scholar

Sawadogo, I., Andre, B., Somda, J., and Zougmore, R. (2017). Evaluating climate change action for sustainable development. Evaluat. Climate Change Action Sustain. Develop. 255–269. doi: 10.1007/978-3-319-43702-6,

Crossref Full Text | Google Scholar

Seydou, T. H., Agali, A., Aissatou, S., Seydou, T. B., Issaka, L., and Ibrahim, B. M. (2023). Evaluation of the impact of seasonal Agroclimatic information used for early warning and farmer communities’ vulnerability reduction in southwestern Niger. Climate. 1–13. doi: 10.3390/cli11020031

Crossref Full Text | Google Scholar

Sidibé, A., Olabisi, L. S., Doumbia, H., Touré, K., and Niamba, C. A. (2021). Barriers and enablers of the use of digital technologies for sustainable agricultural development and food security: learning from cases in Mali. Elementa 9, 1–15. doi: 10.1525/elementa.2020.00106

Crossref Full Text | Google Scholar

Traore, B., Bouaré, Y., Nikoi, G., and Zougmoré, R. B. (2021). Developing public-private partnerships for effective access and use of climate information services by farmers and pastoralists in the great green wall intervention zone of Mali. CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS).

Google Scholar

Traore, B., Corbeels, M., van Wijk, M. T., Rufino, M. C., and Giller, K. E. (2013). Effects of climate variability and climate change on crop production in southern Mali. Eur. J. Agron. 49, 115–125. doi: 10.1016/j.eja.2013.04.004

Crossref Full Text | Google Scholar

Traore, B., Illiasso, A. D., Soulé, M., Arzika, M. Z., Abdoussalam, I., Moussa, A. A., et al. (2024). Low-cost small-scale irrigation for developing an agroforestry system in the semi-arid zone of Niger: case of the apple of the Sahel (Ziziphus mauritiana L.). Front. Sustain. Food Sys. 8:1282235.

Google Scholar

Traoré, B., Ouédraogo, M., Birhanu, B. Z., Zougmoré, R., and Tabo, R. (2018). Utilisation de l’information climatique au Mali_ Manuel technique à l’usage des agents publics et privés du développement rural. International Crops Research Institute for Semi-Arid Tropics. 40.

Google Scholar

Vincent, K., Conway, D., Dougill, A. J., Pardoe, J., Archer, E., Bhave, A. G., et al. (2020). Re-balancing climate services to inform climate-resilient planning – a conceptual framework and illustrations from sub-Saharan Africa. Clim. Risk Manag. 29:100242. doi: 10.1016/j.crm.2020.100242

Crossref Full Text | Google Scholar

World Meteorological Organization (WMO). (2014). 7 bis, avenue de la Paix P.O. Box 2300 CH-1211 Geneva 2, Switzerland.

Google Scholar

Yang, J. M., Yang, J. Y., Liu, S., and Hoogenboom, G. (2014). An evaluation of the statistical methods for testing the performance of crop models with observed data. Agric. Syst. 127, 81–89. doi: 10.1016/j.agsy.2014.01.008

PubMed Abstract | Crossref Full Text | Google Scholar

Zougmoré, R., Partey, S., Ouédraogo, M., Omitoyin, B., Thomas, T., Ayantunde, A., et al. (2016). Toward climate-smart agriculture in West Africa: a review of climate change impacts, adaptation strategies and policy developments for the livestock, fishery and crop production sectors. Agric. Food Sec. 5, 1–16. doi: 10.1186/s40066-016-0075-3

Crossref Full Text | Google Scholar