Assessing the Social Validity of the SAFMEDS Strategy From the Perspective of Teachers and Children

The Say-All-Fast-Minute-Every-Day-Shuffled (SAFMEDS) strategy promotes fluency across several skills and contexts. However, few studies have reported the social validity key stakeholders associate with using the strategy in schools. Assessing social validity may provide us with some insight into factors that may affect engagement, implementation fidelity, and persistent use of the intervention after the termination of a research study. Study 1 details the findings from a survey completed by teachers who have used the strategy in their schools (N = 55). Using thematic analysis, we identified three themes: 1) factors that promote and limit progress, 2) confidence, and 3) inherent advantages of the SAFMEDS strategy. These themes encapsulate teachers experiences of implementing the strategy under the real-word conditions of the classroom and the accompanying advantages and potential challenges they face. Within study 2, we discuss themes arising from interviews with children (N = 26) about their views and experiences of using the SAFMEDS strategy. These children had used the strategy with their teacher for one academic year to promote fast and accurate recall of arithmetic facts. Analysis of these transcripts revealed five further themes relating to children’s engagement with the strategy: 1) enjoyment, 2) data, 3) sense of achievement, 4) skills, and 5) home use. Collectively these themes have potential impact with regards to future training and support models for the SAFMEDS strategy.

Introduction

Traditionally, within educational practice, mastery reflects the ability to perform skills to a level of at least 80% accuracy (Dalton and Hannafin, 1988; Fuller and Fienup, 2018). However, this criterion does not reflect the rate at which learners are able to perform these skills. Binder et al. (2002) highlighted that if classroom tasks enabled children to practice skills beyond traditional mastery, they will be able to recall facts fluently (i.e., accurately and at speed). Once a child can perform a skill fluently, they will be more likely to satisfy four learning outcomes—retention, endurance, stability, and application (RESA; Binder, 1996). Johnson and Layng (1996) identified that children need to acquire fluency of basic skills before progressing onto more complex skills within any subject. Precision Teaching (PT) is a system for defining and measuring fluency of skills on a child-by-child basis (Kubina and Yurich, 2012). By adopting the PT approach, educators make data-based decisions about subsequent instruction based off the learning pictures that emerge on a child’s Standard Celeration Chart (SCC; Binder and Watkins, 1990; White 1986). Lindsley (1995) outlined three categories of learning pictures: improving, maintaining, and worsening. If upon visual inspection, a child had made little or no progress toward their fluency aim (i.e., their data depicts a maintaining or worsening picture) then their teacher may need to alter their instruction, the materials they provided, or the environment. Adhering to the PT approach allows educators to make real-time decisions and prevent prolonged periods of non-progression (Merbitz et al., 2004).

Say-All-Fast-Minute-Every-Day-Shuffled (SAFMEDS) is a practice and assessment strategy that applies the principles PT (Potts et al., 1993). Children typically practice the SAFMEDS strategy using a deck of flashcards; with a question on the front and the corresponding correct answer on the back (Meindl et al., 2013). A child reads the front of the card silently before vocalizing the answer (Quigley et al., 2018). They then receive immediate feedback by checking their answer with the back of the card (Lindsley, 1996). The aim is to get through as many of the cards as quickly as possible, while separating the correct responses from learning opportunities (Cihon et al., 2012). Previous research has demonstrated that, when paired with data-driven decisions, the SAFMEDS strategy can improve skill fluency across several academic domains. For example, learners can use the SAFMEDS strategy to improve recall of arithmetic facts (Casey et al., 2003; Nam and Spruill, 2005; Cunningham et al., 2012; Hunter et al., 2016), words in a second language (Bolich and Sweeney, 1996; Beverley et al., 2016), and subject-specific terminology (Beverley et al., 2009; Stockwell and Eshelman, 2010; Meindl et al., 2013).

The existing literature suggests that the SAFMEDS strategy is suitable for use in schools and can be successfully adapted across a range of school contexts and classroom needs. Practitioners have used the strategy to elicit impactful academic outcomes on a variety of scales from one-on-one practice (Cunningham et al., 2012) to class-wide (Hunter et al., 2016); with both mainstream learners (see, for example, Greene et al., 2018) and with children who have additional learning needs (see, for example, Casey et al., 2003). It should be noted that the outcomes from these research studies were elicited under direct observation of a researcher to ensure higher levels of implementation fidelity. Emerging research has shown that teachers can implement the SAFMEDS strategy independently in their classrooms after training to improve academic outcomes (Beverley et al., 2016). However, fluency gains seem to be greater if a researcher offers coaching to teaching staff after initial training; with Owen et al. (2021) demonstrating that just three 1 h intermittent in-situ coaching visits can positively impact children’s fluency outcomes.

Despite a developing quantitative evidence-base supporting the use of the SAFMEDS strategy in schools, no known studies have robustly evaluated the social validity stakeholders associate with using it. Social validity refers to the perceived importance, acceptability, and viability of a program (Foster and Mash, 1999). Wolf (1978; as cited in Schwartz and Baer, 1991) formally introduced the term “social validity” into the field of behavior analysis to make researchers aware that non-acceptance of interventions by key stakeholders could lead them to reject the program. Even if an intervention has a growing quantitative evidence-base, Thornicroft et al. (2011) identified that stakeholders do not necessarily continue to use interventions after the termination of a research study. It is therefore important that we identify factors that might facilitate or prevent the uptake of the SAFMEDS strategy in schools. To this end, Nelson and Campbell (2017) proposed that researchers should not only consider “what works?” but also “what matters?”. Social validity is a multi-dimensional construct that encapsulates whether 1) an intervention is wanted and needed, 2) stakeholders accept the procedures and consider them feasible to implement, and 3) stakeholders are satisfied with the outcome effects (Foster and Marsh, 1999). Assessing social validity is also important in the context of empowering practitioners to revise and refine educational interventions to promote positive outcomes (Common and Lane, 2017).

Following the use of the SAFMEDS strategy in class to focus on building fluency of basic numeracy skills, approximately 90% of the children in Hunter et al’s (2016) study indicated they would like to continue using the strategy. Moreover, when provided with a choice between Individualized Descriptive and Exploratory Analysis (IDEA) assignments, Think Aloud Pair Problem Solving (TAPPS), and the SAFMEDS strategy, only 26.3% of Adams et al’s (2018) sample rated SAFMEDS as their favorite undergraduate course activity. Whilst these findings provide some insight into people’s attitudes toward the SAFMEDS strategy they did not explore the reasons underlying why some would choose to engage “with it and others” would not. Further anecdotal findings from Beverley et al. (2016) suggest that children enjoy using the SAFMEDS strategy as a method of learning (e.g., they would begin working through timings without a teacher having to ask). Yet, no known research has robustly evaluated factors associated with the social validity of this strategy through qualitative methods. To the authors’ knowledge, no research has also yet to document teachers views on, and experiences of, implementing the SAFMEDS strategy in their classroom. As key stakeholders in the implementation process, addressing the gap in this literature may provide insight into necessary revisions and refinement to support fidelity and positive outcomes.

Qualitative methods such as surveys and interviews enable us to access and understand behavior that is not always directly observable and quantifiable, such as thoughts, experiences, and feelings (Braun and Clarke, 2013). This paper details two qualitative studies focusing on the social validity of the SAFMEDS strategy from the perspective of teachers and children who have used it in schools across North Wales. Specifically, the questions that we asked aimed to gain insight into teachers’ experiences of implementation, perceived usefulness of the strategy, and factors that may affect children’s engagement. Our research aimed to gain insight into the social validity teachers and children associate with using the strategy under the everyday conditions of the classroom, so that we can better understand how to support schools with any unanticipated challenges that they may face (and pre-emptively troubleshoot, if appropriate). We anticipate that this may contribute toward supporting teachers to persist with the implementation of the strategy and maintain/improve children’s engagement. Furthermore, we hoped to be able to reflect upon positive elements and outcomes of the SAFMEDS strategy to ensure that we continue to promote these factors in practice.

Due to the nature of the research questions and the type of questions we asked, we decided to analyze these data separately and present them as distinct studies for two reasons. First, we asked the teachers (study 1) and children (study 2) different questions. While some of the emerging codes might have overlapped, there was no guarantee upon data collection and analysis that this would be the case. Secondly, we used two different research methods to generate the data across the two studies. Teachers’ responses to the survey were unprompted, while the semi-structured interviews required more prompting and exploration during the interviewing process. Finally, we were concerned that combining the datasets might result in some key ideas losing their saliency and, therefore, not accurately reflect the experiences of the two discrete samples.

Study 1: Teachers’ Views and Experiences

Method

Ethics

This study received ethical approval from Bangor University’s ethics committee (application number: 2018-16309). The first page of the online survey stated that we were interested in collecting data relating to teachers’ and teaching assistants’ (TAs) views and experiences of using the SAFMEDS strategy in schools. All participants were invited to provide consent before beginning the survey and had the option to leave questions blank if they did not wish to answer them. Throughout this paper we have not referred to the staff or schools by name to protect their anonymity.

Recruitment

The first author emailed all members of staff who had attended a SAFMEDS training session between the years 2016 and 2018 with a link to the online survey. These training sessions had been organized by the Regional School Effectiveness and Improvement Service for North Wales (GwE) in collaboration with researchers at Bangor University. As such, all staff received training from a researcher to follow the SAFMEDS strategy in line with the procedural steps outlined in Table 1.

TABLE 1

TABLE 1. An outline of the SAFMEDS strategy.

Sample

Fifty-five members of staff from schools across North Wales completed the online survey. Table 2 contains details of the job titles of school staff who completed the surveyand a list of skills that the children in their schools were practising and assessing using the SAFMEDS strategy.

TABLE 2

TABLE 2. Teaching Roles of the Teaching Staff Who Completed the Online Survey and the Skills that they Reported Supporting Using the SAFMEDS strategy.

Procedure

Staff could complete the survey in English (n = 46) or Welsh (n = 9) at their own convenience. We devised the questions for our online survey to broadly capture staff experiences of using the SAFMEDS strategy within their school. These questions were direct in the sense that we asked teaching staff to identify what they perceived the benefits and challenges of the SAFMEDS strategy to be, but we tried to avoid priming any specific responses (for example, we did not ask “did any of the children in your group engage in cheating?”, instead we asked “is there anything that the children do during a SAFMEDS session that you think hinders their performance?”). We believe that this helped us to collect a more holistic view of how the strategy functions under the everyday conditions of a classroom (for a full list of questions, refer to supplementary material). We were unable to pilot the questions before due to the timescale associated with this studentship.

Five questions collected demographic information (i.e., name, school, job title of the respondent, role of the staff member who ran the SAFMEDS sessions, and which skills they targeted using the SAFMEDS strategy). The remaining seven questions aimed to gain insight into the advantages and challenges associated with implementing a SAFMEDS program in schools. We used open-ended questions so that each member of staff could provide as much or as little detail as they wanted in their response to each question. The survey also provided space for staff to justify whether they would use the strategy in the future and to report any case studies that they wanted to share. We translated the Welsh survey responses into English prior to analysis.

Analysis

Researchers can interpret meaning in qualitative data by identifying themes across participants’ responses (Tolich and Davidson, 2003). Thematic analysis is a method that researchers can use to systematically identify and organize qualitative data into themes (patterns of meaning) across the dataset. Crucially, thematic analysis allows researchers to make sense of a targeted group of people’s experiences by identifying commonalities opposed to analyzing data on a person-by-person basis (Braun and Clarke, 2012). The form of thematic analysis that we used for these two studies focused on using an inductive (bottom-up) approach to data coding. This means that the codes and themes emerge from the content of the transcripts themselves (Braun and Clarke, 2006). By following an inducive approach we hoped to report a more holistic overview of the strengths and challenges of using the SAFMEDS strategy as a learning tool, as this is something lacking in the current research literature.

The first author adhered to the six stages of thematic analysis outlined by Braun and Clarke (2006). To become familiar with the dataset she printed out each completed survey and read through each response twice before analysis. This enabled her to become immersed in the data and become familiar with the breadth and depth of the content. On consecutive readings she became more active in the way she responded to the data by making a note of any salient and interesting comments within the dataset in the paper margins, along with some initial thoughts about emerging codes.

Following the immersion phase, the first author worked through each response within the transcript in detail to capture some of the key codes across the data (for an example of this process, refer to Table 3). During this phase, the first author coded as many themes as possible, and retained the surrounding text to prevent any loss of meaning and/or context that might be useful later.

TABLE 3

TABLE 3. An example data extract from the online survey and the corresponding codes.

The first author transferred each code onto a post-it note to allow the identification of candidate themes from the survey responses. The post-it notes acted as a visual aid to help organize and rearrange codes/ideas into broader level themes. This process led to the development of a thematic map (Figure 1). This map depicts the final themes, subthemes (where appropriate), and their relationship to one and other. Once the candidate themes had been refined and checked against the origional data extracts, the first author named each theme and built the narrative presented within the result section of this paper. We acknowledge that the reliability of the process would have been enhanced if an additional researcher independantly coded the data for comparison. However, this research was conducted as part of a PhD studentship so funding and resources were limited. Each author read the manuscript and supporting quotes to confirm the clarity of each theme and to ensure a cohesive narrative of the results.

FIGURE 1

FIGURE 1. Thematic map of the themes within the online survey dataset.

Results

Theme 1: Factors That Promote and Limit Progress

Setting Up the Program

The staff explained that setting up a SAFMEDS program in schools requires a transitionary period, whereby the children need to get used to using the cards and following the prescribed stages of the strategy. Following this phase, the children are typically able to take ownership of their own learning and participate in the sessions with minimal input from an adult.

Good structure and program runs itself once the children are aware of the routine (Deputy headteacher).

A big factor of the success is giving the intervention time. Once the children get into the routine they can do the session with minimal adult support (HLTA).

It was difficult at the beginning for them to use the cards–they would drop them or mix them up with the person sitting next door to them. Now this is not an issue as they are used to dealing with the cards (Deputy headteacher).

Staff also highlighted the importance of embedding the program into the day-to-day timetable of the school. Holding the SAFMEDS sessions on the same days each week, at the same time, and in the same place helped staff to ensure that the children could regularly engage with the strategy. Without this routine in place, staff reported that they would miss sessions, other events would take priority, and the children would make limited progress.

Sessions held at the same time, and in the same place by the same person each day. Quiet room–pupils were responsible for recording their own results (Deputy Headteacher).

Pupils sometimes miss out on SAFMEDS that day which hinders the speed they progress (TA).

When considering the logistics of a SAFMEDS program, several members of staff commented on advantages of working with smaller group sizes. They reported a preference for running the intervention on a one-on-one or small group basis to ensure that the children receive appropriate attention and support. This was a particularly common view of staff who reported targeting children with additional learning needs. Five teaching staff also indicated that the intervention could be scaled-up to work with larger groups once the children are able to follow the prescribed steps independently.

We would consider using SAFMEDS in the future, but in a different way to how it's used now. I'd like to see an example of how SAFMEDS is implemented across a year group, delivered as a whole teaching session as opposed to 1-2-1 (Teacher).

Yes, I would [use SAFMEDS again], but I would like to use it in smaller groups as I find that in a whole class environment the pupils get frustrated with the waiting for others (Teacher).

To achieve the best results from the program, staff reported that it was important to have an appropriate environment to run the SAFMEDS sessions in. This included having a designated quiet space for the children to work in. In schools where space was limited, staff found a reduction in children’s ability to concentrate due to surrounding events (e.g., other children walking between classes).

The regular number practice in a quiet environment helped the success of the SAFMEDS session (Teacher).

Distractions (can hinder the success of a SAFMEDS session) as we work in the corridor, children walk past and often classes go out which can be noisy (TA).

Personal Characteristics

Children’s attitudes toward each session played a role in the overall success of the SAFMEDS program. High levels of concentration from the children helped them to engage with the timings and improve their scores. Many of the teaching staff highlighted that the children perceived the sessions to be enjoyable, which in turn motivated the children to attend and engage with the strategy.

The children have enjoyed attending the sessions, and we have seen a marked improvement in their confidence and ability (Deputy headteacher).

The pupils are thoroughly enjoying their SAFMEDS sessions, they are learning while having fun. Their ability to answer multiplication facts quickly without having to calculate the answer is definitely improving already (Deputy headteacher).

The children have responded positively to the sessions and are visibly enthusiastic about attending (TA).

Attendance

Many of the staff ran the intervention with smaller groups of children, which often meant withdrawing them from class to complete the SAFMEDS sessions. Staff indicated that some children would refuse to attend if there was a competing, more appealing, activity available in the classroom. Moreover, other factors such as staff availability and the children being absent from school affected attendance to SAFMEDS sessions.

(Children will not come out for the SAFMEDS session because there is a) fun activity for rest of class OR worry they won’t finish their work in class or miss out on learning if they come out: both cause resistance and annoyance toward SAFMEDS (TA).

Other events happening in the school so that the room was unavailable or TA having to cover another TA in class. Pupil absence could be a factor (that hinders progress) too (Deputy headteacher).

Cheating

Due to the self-directed nature of the SAFMEDS strategy, staff noted that some children cheated during the sessions. Staff provided the example of children placing cards in their “correct” pile that they had either answered incorrectly or not at all. Staff were able to identify that cheating hindered the children’s progress. In a group scenario, one TA noted that some children engaged in cheating to avoid being teased by their peers.

Keeping an eye on children that place cards in their yes box they may not have necessarily known the answer! This was better when mentors started to be involved (TA).

Cheating took place at the start that has been nipped in the bud. At the start there was the “I scored better than you situation” where the children teased anybody that scored low again that has been nipped in the bud (TA).

Sometimes if the child isn’t as confident as another or not as quick it can put them off or “cheat” so they can put extra cards in the yes pile (TA).

Impact of Competition

Self-competition resulted in children setting individualized targets and monitoring their scores across sessions. Staff reported this as a good method of promoting healthy competition, which helped keep the children engaged and stay motivated. Some members of staff reported that some healthy peer competition helped children to improve their scores across sessions.

It is also good for them to see the progress they have been making throughout the session and also throughout the number of weeks we have been running SAFMEDS. This gives them an incentive to want to improve and get more answers correct next time (Teacher).

A couple of my group have low self esteem and I think that they have gained confidence when they see their results improving. There is a healthy competitive spirit between a couple of them (HLTA).

(Child name) who is very nervous and had no confidence didn’t like it at the start, but as the sessions progressed, he made a good leap in his scores and now dances around when he surpasses his last best score. It is cute to watch him counting how many he got right and then to see the delight on his face when he realizes he beat his last best score (TA).

Peer competition within groups fostered a negative attitude toward the SAFMEDS sessions for some children. In some cases, this led to bullying behavior toward children achieving low scores. Some of the survey responses also alluded to feelings of inadequacy if a child did not receive the highest score within the group. Peer competition also appeared to influence the children’s decision to cheat by writing down a higher score.

As in any group children become obsessed with doing better than other children in the group. Making fun of children who under perform (TA).

The competitive nature of the program can also have a negative impact on the pupils as they can get upset if another pupil achieves a higher score (Deputy headteacher).

If two characters have had a fallout; competing over who has the better score when they were all on the same pack put some children down; (the children started) turning cards slightly early to beat their score (TA).

Theme 2: Confidence

The survey responses revealed that children’s confidence improves throughout a SAFMEDS program; particularly with regards to the skill(s) that they had been practicing. As a result, children become less anxious about answering cards incorrectly and feel confident enough to use their knowledge outside of SAFMEDS sessions. Staff indicated that they had observed the children exchanging their new knowledge with other teachers and being more willing to answer questions in class.

The children have enjoyed attending the sessions, and we have seen a marked improvement in their confidence and ability (Deputy headteacher).

I think their confidence increased in that subject, but also increased their independence and I often see the children passing on their SAFMEDS knowledge to other children. Before SAFMEDS I don’t think any of the children who took part would have willingly guided other children (HLTA).

(The children are) more confident in class maths lessons - more willing to put hand up in class (TA).

Theme 3: Inherent Advantages of the SAFMEDS Strategy

It is easy for children to learn how to follow the steps of the SAFMEDS strategy. Additionally, each timing takes 1 min to complete which makes the procedure quick to implement. Once in a routine, staff found the procedure to be convenient to use with the children they were supporting.

Speed and convenience make this an easy resource to use (Headteacher).

It is a short burst of intervention, its quick paced, you can see the instant response. The children revisit the same skill lots of times (HLTA).

Developing fluency is the goal of a SAFMEDS program. As such, staff reported a noticeable difference in the speed and accuracy in which children could perform skills. In schools where the target was to improve fluency of basic arithmetic facts, staff reported that children’s mental recall of these facts improved across the intervention.

Overall, I have seen a marked difference in the children's confidence and fluency with single digit addition (Teacher).

Improved the speed and accuracy of addition and subtraction in some children (TA).

The children have improved in their number recall and number facts which has been noted on by their teachers (Teacher).

As a result of improved fluency in prerequisite skills, children were able to progress onto more advanced packs (e.g., move from practicing number recognition to single digit addition sums). Staff also reported that children developed other complementary skills, such as problem solving. Due to the inherent advantages of the SAFMEDS strategy, all but three members of staff indicated that they would like to use the strategy in future. Two members of staff did not respond to this question, and the remaining teacher stated that the intervention was too time consuming to align with their school schedule.

Pupil confidence has improved in terms of number recognition and number bonds. They are able to apply these in lessons in class and parents have reported that pupils are talking about maths in a more positive way (Deputy headteacher).

Pupils are more willing to respond in mental mathematics lessons. Pupils’ problem solving work improved as a result (Teacher)

(Child’s name) is now able to recognize most numbers independently. Her confidence with single digit is also growing (Teacher).

(A child has) gone from +1 to +2 to now having gone through the whole single digit addition pack is working on the subtraction pack (TA).

Discussion

Previous research has demonstrated that researchers and teachers can implement the SAFMEDS strategy in school settings to promote fluency in several subjects (e.g., Casey et al., 2003; Beverley et al., 2016). Yet, no known research has investigated the social validity of the strategy from the perspective of teachers. To gain insight into this gap in the literature, we conducted a qualitive study evaluation of teachers’ views and experiences of using the SAFMEDS strategy with the children in their school. By analyzing the survey responses, we were able to identify three themes broadly relating to the advantages and challenges of using the strategy in schools.

Thornicroft et al. (2011) identified that researchers need to identify factors that may facilitate or prevent stakeholders from using an intervention in their day-to-day practice after initial implementation. The outcomes of the present study suggest that teaching staff may initially struggle to implement the SAFMEDS strategy with children in their classroom. Staff identified that setting up a SAFMEDS program requires a transitionary period during which children need to familiarize themselves with how to handle the cards and staff need to establish a routine for the SAFMEDS sessions. This is important to consider in terms of initial and maintained fidelity of implementation. Westfall et al. (2007) suggested that some manualized interventions do not align with the day-to-day challenges within real-world settings due to the confounding variables (e.g., competing internal school events) that are not present in more controlled research studies. This can sometimes result in educators modifying intervention protocol (Glasgow et al., 2003). Lack of adherence to the manualized steps of an intervention program can result in smaller effect sizes across academic outcomes (Durlak and DuPre, 2008). This finding should empower researchers and practitioners to consider how we can best support teaching staff during the initiation and maintenance of a SAFMEDS program; particularly with regards to helping staff maintain high levels of fidelity during the initial transitionary period and ensuring that they persist with its use beyond the first few sessions.

Two of the dimensions of social validity described by Foster and Marsh (1999) are that stakeholders accept the procedures and consider them feasible to implement and they are satisfied with the outcomes. Despite some initial challenges setting up the program, staff identified that children developed independence as the weeks progressed. Consequently, the children were able to participate in the program with minimal adult supervision and still gained fluency in the skills they were practicing. This suggests that beyond the transitionary period, staff found the SAFMEDS strategy feasible to implement. Moreover, the strategy promotes two beneficial outcome effects: independence and increased fluency.

Graf and Auman (2005) promoted the idea that children should take ownership of their own learning during a SAFMEDS program and, whenever possible, children should hold their own cards and record their own data. Staff indicated that an associated benefit of this independence could be scaling up the sessions with time (e.g., from small groups to class-wide). While the SAFMEDS strategy has potential in terms of scalability, teaching staff need to be wary of children cheating. If children are recording their own data, a level of trust from the member of staff is necessary to ensure that they are recording the correct score. Staff who participated in this study highlighted that some children resorted to cheating, particularly when they delivered the program on a group or class-wide basis. There are no known empirical reports of children cheating during a SAFMEDS intervention This could be due to the focus on single-case design research, whereby it would be much more difficult for cheating to go unnoticed. However, Vargas (2013) explained children might cheat during an academic intervention to avoid punishment (e.g., being perceived as less able in front of their peers or teacher). Further investigation is necessary to reduce punishment contingencies within a SAFMEDS program to prevent unhealthy peer competition and help children conceptualize low scores in a more positive way.

One method of fostering positive competition is to shift children’s attention toward meeting and/or exceeding their own targets. Staff in this study reported that self-competition motivated children and kept them engaged with the task. Wyse (2001) explained the importance of setting targets that are not too difficult to achieve, and that staff should discuss the importance of setting targets orally to ensure children understand why they should engage with them. Children should also reflect on their progress frequently. Following each SAFMEDS session, children were advised to plot their best score on their SCC (Eshleman, 1985), and the data on a child’s SCC provides a visual representation of their progress over the intervention period (White, 1986). Using SCCs can be a simple and effective way of setting achievable session-by-session targets for each child. Moreover, staff should consider displaying a child’s overall fluency aim for the skill on the chart. This will help the children reflect on where they have started from and what they are aiming to achieve (White and Neely, 2012). Revision and refinement of the strategy in this way can help promote positive outcomes, which is an added dimension of social validation (Common and Lane, 2017).

Study 2: Children’s Views and Experiences

Method

Ethics

This study received ethical approval from Bangor University’s ethics committee (application number: 2018-16309). We monitored the children’s assent throughout the interview. If any child indicated that they did not wish to participate, the first author would have terminated the interview and informed a relevant member of staff. Also, if there were any questions that a child did not want to answer, the first author would have allowed them to move onto the next question. None of the children or their schools are identifiable within this paper to protect their anonymity.

Recruitment

We sent out an email to schools who had sent teachers and/or TAs to attend a SAFMEDS training session in North Wales between 2016 and 2018. Following an expression of school-level interest, we disseminated opt-in consent forms to all the parents/guardians of children who were participating in a SAFMEDS program at the time of recruitment. This form explained that the first author would run a one-on-one interview with their child to ask them to demonstrate a SAFMEDS timing, and ask about their experiences of using the SAFMEDS strategy.

Sample

All the children had participated in a SAFMEDS program during the 2017–2018 academic year (October to July). This intervention focused on promoting fast and accurate recall of arithmetic facts. We received consent to interview 26 children who attended a primary school in North Wales (see Table 4).

TABLE 4

TABLE 4. Demographic characteristics of the children.

Procedure

The first author conducted one-on-one semi-structured interviews with children who had used the SAFMEDS strategy with a member of teaching staff in their school. By nature, semi-structured interviews begin with pre-defined questions, but the interviewer can ask additional un-scripted follow-up questions. Prompting for elaboration during a semi structured interviews can be an invaluable tool for ensuring the reliability when working with children; allowing for clarification of any interesting and relevant issues raised (Hutchinson and Wilson, 1992). The semi-structured approach also afforded the first author the opportunity to rephrase the question if a child did not demonstrate understanding, ask additional questions for clarity or expansion, and explore additional relevant information as the interview unfolded.

We opted to run one-on-one semi-structured interview for this study for two reasons. First, the children completed an icebreaker activity to allow them to demonstrate how they used the cards. This enabled the first author to assess if they had been correctly following the procedural steps of the SAFMEDS strategy with their teacher. It would have been difficult to collect fidelity data for every child in a group setting. Second, engaging in a one-on-one interview with the children enabled the first author to gauge their personal experience of using the strategy—reducing any social influence from their peers.

The first author conducted the interviews in a quiet space within the children’s school. As an icebreaker activity, she asked each child if they could demonstrate how they used the SAFMEDS cards with their teacher. She provided each child the deck of skill-appropriate SAFMEDS cards (based on the skills that they had been practicing), a placemat (so they could easily separate their cards into two piles—corrects and learning opportunities), a table to record their scores, and a SCC to plot their best score onto. The children demonstrated a 1 min SAFMEDS timing, how they engaged in an error correction procedure, and how they recorded their own data. The first author also asked the children how many timings they typically completed per session and how many times a week they came out of class to complete SAFMEDS sessions. As the children completed this activity, the first author made a note of whether they completed each of the procedural stages listed in Table 5. Due to the age of the learners, the researcher did not explicitly tell the children that they were being scored for this activity to reduce any associated anxiety; the researcher simply indicated that she knew that they were using the SAFMEDS strategy with their teacher and asked if they would demonstrate it for her. There were no instances where a child showed signs that they were uncomfortable with the processes. Had any child had indicated that they did not want to take part, the first author would have acknowledged this and asked if they wanted to end the session and return to class. Only after she was satisfied that the children were not making any fundamental mistakes that compromised the integrity of the SAFMEDS strategy would she ask if each child was happy to answer some additional questions.

TABLE 5

TABLE 5. The fidelity checklist we used to establish the extent to which the children engaged with the procedural stages of the SAFMEDS strategy.

Following the icebreaker activity, the first author asked the children three predetermined questions relating to their experience of using the SAFMEDS strategy. These questions focused on whether they enjoyed using the strategy, if they thought the strategy was useful, and if they use (or would like to use) the strategy outside of school. When appropriate, the first author asked the children elaborate on their answers to extract more detail. Refer to the supplementary material for a full list questions and prompts that the first author asked during the semi-structured interview. Again, we were unable to pilot the questions before due to the timescale associated with this studentship. However, we hoped that the children’s answers to these questions would help provide the foundation of further questioning in future (e.g., exploring any unexpected challenges/advantages in more depth and across different age groups).

Analysis

We applied the same analysis protocol detailed in study 1 to evaluate the interview transcripts. As an additional familiarization step prior to analysis, the first author transcribed each interview from a Dictaphone recording. Figure 2 displays the resulting thematic map.

FIGURE 2

FIGURE 2. Thematic map resulting from the children’s interview transcripts.

Results

Fidelity

The children adhered to most of the procedural stages outlined in Table 5 during the icebreaker activity (M = 92.86% of stages; range: 71.43–100%). Table 6 outlines the procedural errors made by the children in each school.

TABLE 6

TABLE 6. Percentage of SAFMEDS steps the children in each school adhered to.

Theme 1: Enjoyment

We asked the children if they enjoyed taking part in SAFMEDS sessions with their teacher. All the children in this sample indicated that they enjoyed using the strategy. However, few were able to provide justification surrounding particular elements that they enjoyed or why they enjoyed it. The transcripts revealed that the children saw SAFMEDS as a fun strategy to use. Two of the children also identified that they enjoyed mathematics before they started using the SAFMEDS strategy to practice their skills, which by proxy made the activity more appealing.

(Participant 22:) now there is just me and (names) and that and it’s just so much fun!

(Participant 1:) It’s a lot of fun.

(Participant 5): (I enjoy) when you practice the cards.

Theme 2: Data

All the children who participated in this study noted down their scores on a recording sheet, with many of them transferring their best score for the day onto their SCC. Across the transcripts, there was a clear sense that children were motivated by counting the number of cards they answered correctly, particularly if they had exceeded their personal best score. By charting their highest score each day, the children were able to visualize their scores increasing.

(Participant 21:) You get to go up on the chart. (KO:) you like using the chart do you? (Participant 21:) So much!

(Participant 26:) It’s enjoyable that you get better scores

(KO:) What is your favorite part about using SAFMEDS? (Participant 13:) when we get the higher score

Theme 3: Sense of Achievement

This theme is closely related to the children’s attitudes toward collecting their scores but focusses on their associated perceptions and motivations. Five comments within the transcripts showed that children associated the strategy with making them more intelligent. These children used words such as “clever” and “smarter” to describe how the SAFMEDS strategy contributed to making them feel.

(Participant 10:) (I use SAFMEDS) because I get even better at learning. I get smarter.

(Participant 24:) Because I wasn’t very good but now I’m getting better and better. My mum even sees a difference and my dad is really good at maths and he gives me some maths questions sometimes. I work them out and then most of them are all right.

The children who attended school 3 also expressed that they were motivated by the achievement they felt because of their teachers’ approval. They were keen to develop their fluency so that they could show her their progression. This motivated them to want to engage with further practice outside of the timetabled sessions within school.

(KO:) Would you like to use it outside of school if you could? (Participant 23:) *nods* (KO:) Why would you like to do that? (Participant 23:) So I could show Miss (name) how well I did.

One way in which the children felt that they were showing improvement was by answering more cards correctly than they had achieved on previous attempts. As such, several comments alluded to the achievement that the children felt when they were able to answer all of the cards in the pack correctly within 1 min. Also, these children recognized that since taking part in the SAFMEDS program they had become more fluent at recalling arithmetic facts. Achieving fluency appeared to encourage children to engage with the strategy, both in and outside of school.

(KO:) why would you like to do (the SAFMEDS strategy) at home? (Participant 1:) Because I can learn better when I go to school. I can do it will get more faster.

Theme 4: Skills

All of the children alluded to the skills that they had developed throughout the course of their involvement with the SAFMEDS program. We identified two sub-categories relating to skill development: learning skills and independence.

Learning Skills

We asked the children if and why they thought the SAFMEDS strategy was useful. Many of the children identified that the strategy helped them to improve their mathematics skills, including supporting their fluency of arithmetic facts. Some children went on to explain that they had worked through multiple packs across their involvement with their SAFMEDS program, as such they were able to answer progressively more difficult mathematics questions.

(KO:) yeah? What do you enjoy about (using the SAFMEDS strategy)? (Participant 11:) It helps me with maths.

(KO:) okay, what is your favorite part about doing the cards? (Participant 2:) Where I swap over cards (KO:) Do you mean when you move onto different packs? (Participant 2:) yeah.

(KO:) What do you enjoy about (using the SAFMEDS strategy)? (Participant 24:) Well, last year I wasn’t good at maths because I wasn’t very good at like adding stuff and taking away. (teachers name) came to collect me. she told me what I was doing and then we went on to the adds then, like a couple of months later I went onto the take aways and then I went onto the adds and take aways.

Independence

As well as the academic gains stemming from the SAFMEDS strategy, a prevalent theme among the transcripts was the independence that the strategy afforded children. The majority of children we interviewed were keen to use the cards independently, even if support from an adult was accessible (such as from a parent).

(KO:) Would you use them on your own or would you use them with your mum and dad? [Participant 7:] On my own.

(KO:) Do you do them on your own or with your parents? (Participant 10:) I do it on my own because I have a timer on my PlayStation.

(KO:) What’s your favorite part about doing SAFMEDS? (Participant 17:) That you get to fill in your own sheet.

Theme 5: Home Use

Eight children claimed that they already used the cards to practice using the SAFMEDS strategy at home. A further 14 children identified that they would like take the cards home for practice. The children who participated in these interviews saw the advantages of practicing the cards regularly as it helped them to develop their mathematics skills at a faster pace than just practicing at school.

(Participant 2:) Is there an app on the computer you can get (SAFMEDS) on? (KO:) No, not yet. Would you like an app? (Participant 2:) Yeah. (KO:) Why would you like an app? (Participant 2:) Because I can use it outside of school and in school.

(KO:) Do you use SAFMEDS at home? (Participant 15:) *nods* (KO:) Why do you use them at home? (Participant 15:) To get better at this.

(KO:) Why would you like to use (SAFMEDS) at home? (Participant 13:) So then I learn every day.

While the majority of the children we interviewed claimed to want to use the cards independently, eight children identified that they would like to involve their parents in the strategy if they could take the cards home. Examples of the support parents could offer included holding their child’s cards during the timing and helping their child to address any cards in their “not yet” pile (via error correction).

(KO:) Do you like having your parents helping you (to do the SAFMEDS strategy)? (Participant 5:) Yeah. (KO:) Why? (Participant 5:) Because it helps. (KO:) How do they help you? What do they do? (Participant 5:) They hold the cards and then I said the answer and they put it down.

(KO:) Would you do (SAFMEDS) by yourself or with someone else? (Participant 24:) I would do them with somebody else, but they would give me the timer and then I would make my own board, probably. Then, my mum will probably time me on the timer. When it’s stopped, I will probably tell my mum and then she’ll do my not yets.

Three children suggested that other activities available to them around the house reduce the appeal of using the SAFMEDS strategy at home. These children still claimed to enjoy using the strategy with their teacher but would opt to leave the flashcards at school.

(KO:) Would you like to use them at home more? (Participant 4:) No (KO:) No? Why not? (Participant 4:) Because we have loads of stuff to do at our house.

Discussion

Previous research has demonstrated that the SAFMEDS strategy can help children to improve their mathematics skills (e.g., Hunter et al., 2016; Greene et al., 2018). However, no known research has robustly evaluated the social validity children associate with the strategy. This study aimed to provide some insight into primary-aged children’s experiences of using the SAFMEDS strategy with their teachers in their school. We were successful in identifying five themes; each demonstrating some benefits of using the SAFMEDS strategy to support development of mathematics skills. Specifically, these themes provided a broad insight into factors that might affect children’s engagement with the strategy.

Approximately 90% of the children in Hunter et al. (2016) sample (N = 19) indicated that they would like to continue using the SAFMEDS strategy in school to improve their mathematics skills. However, they did not explore the reasons underlying why some children would continue to engage with it, while others would not. This highlighted the need for a research study assessing the social validity of the SAFMEDS strategy to provide evidence of its acceptability to aid fluency development. All the children we interviewed indicated that they enjoyed using the strategy, with many also indicating that they would like to engage in further practice at home. Moreover, many of the children we interviewed were able to identify the progress that they have made across their engagement with the SAFMEDS program (with particular emphasis on their developing mathematics skills and independence).

The children we interviewed for this study demonstrated a preference for independent practice. Together with the results from study 1, our findings suggest children can take ownership of the SAFMEDS strategy, which in turn promotes independence and establishes a routine over the school year. This concurrent finding adds further credibility to this facet of social validity. An additional advantage of promoting high levels of independence found during this study is that children can engage with the SAFMEDS strategy in any environment that they choose to (e.g., in school or at home); with 22 children identifying that they would like to, or already do, use the strategy at home. This suggests that the intervention is valued by the children who use it, which is one of the dimensions of social validity highlighted by Foster and Marsh (1999). There is currently a lack of published research reporting the effects of children using the SAFMEDS strategy at home to support their learning. Further research in this area would help validate children’s willingness to engage with the SAFMEDS strategy outside of school and evaluate the educational outcomes associated with additional practice.

Some children did identify that they would like some support from an adult when using the SAFMEDS strategy. Examples included providing support during error correction and someone manipulating the cards for them during the SAFMEDS timing. While providing additional instruction to teach new skills during error correction is an imperative part of the learning process, those supporting should be wary of providing too much support during the SAFMEDS timing itself. Binder (2003) explained that certain teaching methods can prevent children from progressing skills at their own pace. Educators should design programs that lift ceilings on children’s performance to enable them to achieve levels of fluency that ensure retention and application to other skills (Binder, 1996). Lindsley (1996) explained that a learner’s rate of responding decreases by approximately half if someone else holds and directs the cards for them, compared to self-held performance. This is due to learners being able to present the next card to themselves twice as fast as a partner could. The results from our study suggest that most children enjoy the independence that the SAFMEDS strategy can afford them, so it is worthwhile exploring how to harness this within future training and research.

A cardinal feature of the PT approach is the use of a SCC to plot and visualize data (Calkin, 2005). Using a SCC allows for assessment and intervention to occur concurrently; making it a useful tool for practitioners to make real-time, data-based decisions (Aninao et al., 2015). From a child’s perspective, the chart provides a way of visualizing their progress. The children we interviewed emphasized the motivational power that data can have on their engagement with the SAFMEDS strategy, particularly with regards to setting themselves a target in each session (i.e., beating their previous personal best score). Applying a pragmatic approach to individualized goal setting using trajectory lines on a SCC might help keep children focused and working toward attainable targets (Johnson and Street, 2012).

We interviewed primary school children for this study. However, children’s views on the strategy might vary depending on their age and the educational institute that they attend (e.g., primary or secondary school; higher or further education). An extension of this research might investigate learners’ views on the SAFMEDS strategy at different stages of education. For example, we could consider why secondary school children might choose to use SAFMEDS as a revision strategy for their pending statutory exams. This extension would provide a more holistic overview of learners’ views of the SAFMEDS strategy.

General Discussion and Conclusion

Social validity refers to the perceived importance, acceptability, and viability of a program (Foster and Mash, 1999). By gaining insight into how the SAFMEDS strategy works under the day-to-day conditions of the classroom, we can empower researchers and educators to: 1) revise and refine the strategy, 2) better understand how to support any challenges that stakeholders may face with implementation and, in doing so, support with persistent implementation, and 3) help maintain/improve children’s engagement. Moreover, by reflecting upon the positive elements associated with using the strategy, researchers can harness this information to improve the quality of training and implementation support.

The SAFMEDS strategy is designed to help learners develop fluency and to highlight areas where they might need additional support. The staff and children who participated in this research acknowledged that the SAFMEDS strategy aids fluent recall of arithmetic facts; with many of the children progressing onto more difficult mathematics skills throughout their involvement with the program. Additionally, children reported that they enjoy using the SAFMEDS strategy to improve their basic mathematics skills, a feature that was validated by teaching staff who noted the visible enthusiasm that children displayed when attending SAFMEDS sessions.

As well as improving academic skill development, children also showed an appreciation for the independence that the strategy afforded them. This is a positive feature in the context of scaling up the program and increasing practice opportunities (e.g., they could practice at school and home). Several members of staff were keen to scale-up the intervention beyond small groups to help support more children in their school. They felt that this would be feasible once children were able to engage with the strategy independently. However, staff should be wary of children cheating when scaling up the program to larger groups of learners.

One way to help reduce cheating within larger groups might be to set the children personalized and attainable goals for each session. Staff reported that this tactic shifts the focus away from unhealthy peer-competition toward self-competition. The children reiterated that they were motivated by beating their personal best scores each session. By making children’s progress visual on a SCC, it might be possible to set attainable goals based on their performance across previous sessions.

This paper provided some useful insight into stakeholders experiences of using the SAFMEDS strategy in schools. It may prompt a useful discussion with regards to the challenges that teachers may face while implementing the strategy, as well factors that might promote/hinder children’s engagement and performance during sessions. However, we acknowledge some limitations associated with the current study. First, the research methods we used might have influenced the themes that we were able to identify. Using an online survey to collect staff’s views enabled us to recruit a larger sample compared to a focus group (Van Selm and Jankowski, 2006) but may have restricted the richness and depth of responses. Using surveys meant that we were unable to ask for elaboration if a teacher and/or TA had provided an interesting response that could have led to further themes.

Kortesluoma et al. (2003) explained that using interviews with children can be a favorable method of collecting qualitative data over surveys because it allows them to have space and time to discuss their experiences, without limiting their responses to a narrow range of categories. Moreover, interview responses are not bound by written comprehension abilities. We opted to interview the children to overcome these two issues and generate what we thought would result in a richer and more insightful dataset. However, it is possible that running the interview on a one-to-one with the researcher prompted the children to provide responses that they deemed to be socially desirable. Although all the children we interviewed indicated that they enjoyed using the SAFMEDS strategy, we acknowledge that this might be an over-representation. It might have been possible to identify additional information about elements of the strategy that children find difficult or unenjoyable if we had employed a research method that afforded a level of anonymity.

We also acknowledge that qualitative research does not happen in a vacuum without potential biases. The research that we design and the questions that we ask are, in part, driven by both who we are as a person (personal reflexivity) and what we know from our own experience of theory and practice (epistemological reflexivity). As such, researchers should reflect upon the ways in which they may have unintentionally implicated the research and its findings (Willig, 2008). Reflexivity does not discredit the usefulness of the findings of qualitative research. Rather, this exercise serves to add a layer of transparency to the research process, in absence of an independent coder. Tuval-Mashiach (2017) suggested that researchers consider three reflexive statements: ‘why we did it’, ‘what we did’, and ‘how we did it’.

We have outlined what we did (research aims) and how we did it (methods) above. In terms of why, the first author had several years’ experience of using the SAFMEDS strategy in schools. Prior to developing this study, the first author had been working closely with teachers to support implementation and had noticed some recurring challenges that they faced after initial training. She was interested in turning some of these anecdotal observations into a more robust evaluation of teacher’s/TA’s views and experiences of using the SAFMEDS strategy in schools. When developing the questions, the authors were keen not to prime specific responses (e.g., relating cheating and use of the SCC) in case this deflected from some of the other challenges that teachers faced when implementing the program. Additionally, they wanted to capture whether teachers felt there were associated benefits with using the SAFMEDS strategy. When working with children in schools, the first author had noticed that some particularly enjoyed using the SAFMEDS strategy (e.g., they would be excited to demonstrate or share their scores), while others were more difficult to engage (e.g., they would engage in cheating tactics or ask to do something else). This provided the initial motivation to conduct a more robust evaluation of children’s experiences and views regarding the SAFMEDS strategy. When designing the questions, the authors were conscious of the age of the children in the sample, so started off with a yes/no statement to help facilitate the discussion (e.g., “do you enjoy using SAFMEDS?”). Once the children responded, this prompt was followed with an open question (e.g., “why do you enjoy it?”). Ultimately, we wanted to gain insight into some of the underlying factors that might affect children’s engagement with the strategy.

The studies that we have presented in this paper provide a solid foundation for further enquiry surrounding the social validation of the SAFMEDS strategy by key stakeholders. While standardized quantitative surveys can help us gain broad insight into stakeholder’s attitudes of an intervention, they often lack the detail necessary to determine some of the key elements of social validation outlined by Foster and Marsh (1999). These include understanding why stakeholders think an intervention is wanted or needed, whether they accept the procedures and find them easy to implement, and if they are satisfied by the outcomes achieved following implementation. Qualitative methods enable researchers to delve into some of the details underlying stakeholders’ responses as they allow for elaboration. Patton (1999) asserted that qualitative enquiry can gain credibility by using rigorous techniques for data collection and analysis; with researchers paying attention to issues of validity, reliability, and triangulation. In this paper, we have outlined the methods that we used for each study and followed the same inductive thematic approach to analysis for each data set. Patton (1999) explained that the logic of triangulation is based on the premise that different research methods, analyses, or samples may provide additional validation of themes or reveal new information. Further extensions of this research could trial these questions through different research methods or data sources (e.g., compare observational and interview data, or compare how people respond on an anonymized survey and a one-one-one interview), re-administering the same research methods to validate the consistency of the results across the same or different samples, or employing an independent coder to reduce any bias during the analysis itself.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors upon reasonable request.

Ethics Statement

This study received ethical approval from Bangor University’s ethics committee (application number: 2018-16309). All teachers provided consent before completing the questions they wished to answer within the survey. We obtained school-level and parental consent to interview the children who participated in study two. We monitored the children’s assent throughout the interviews.

Author Contributions

This work was undertaken as part of KO’s doctoral degree, as such she conducted the interviews with the children, transcribed the interview recordings, undertook the qualitative analysis for both studies, and produced the initial draft of the manuscript. RW, MB, and JH supervised the research; providing input and guidance with regards to the study aims and design. All authors have contributed to edits to the manuscript and approved the submitted version.

Funding

This work was supported by the Economic Social Research Council Doctoral Training Partnership (ESRC-DTP) under the grant number ES/P00069X/1.

Conflict of Interest

KO has received occasional fees to deliver SAFMEDS training to schools in the North Wales region and other parts of the United Kingdom. RW works for the Regional School Effectiveness and Improvement Service for North Wales (GwE), who promoted this project and organizes regular SAFMEDS training for schools across the region.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

A section of this manuscript has been published online as part of KO’s doctoral thesis (Owen, 2020). We would like to thank all the teachers, and children who participated, the Economic and Social Research Council Doctoral Training Partnership for funding the PhD project, and the Regional School Effectiveness and Improvement Service for North Wales (GwE) for their support throughout.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2021.681952/full#supplementary-material

References

Adams, O., Cihon, T. M., Urbina, T., and Goodhue, R. J. (2018). The Comparative Effects of Cumulative and Unitary SAFMEDS Terms in an Introductory Undergraduate Behavior Analysis Course. Eur. J. Behav. Anal. 19 (2), 176–194. doi:10.1080/15021149.2017.1404394

CrossRef Full Text | Google Scholar

Aninao, T., Acevedo, D., Newsome, K., and Newsome, D. (2015). Putting the Precision in Precision Teaching: Using the Standard Celeration Chart for Ongoing Assessment. Behav. Development Bull. 20 (2), 168–172. doi:10.1037/h0101312

CrossRef Full Text | Google Scholar

Beverley, M., Carl Hughes, J., and Hastings, R. P. (2009). What's the Probability of that? Using SAFMEDS to Increase Undergraduate success with Statistical Concepts. Eur. J. Behav. Anal. 10 (2), 235–247. doi:10.1080/15021149.2009.11434321

CrossRef Full Text | Google Scholar

Beverley, M., Hughes, J. C., and Hastings, R. P. (2016). Using SAFMEDS to Assist Language Learners to Acquire Second-Language Vocabulary. Eur. J. Behav. Anal. 17 (2), 131–141. doi:10.1080/15021149.2016.1247577

CrossRef Full Text | Google Scholar

Binder, C. (1996). Behavioral Fluency: Evolution of a New Paradigm. Behav. Analyst 19 (2), 163–197. doi:10.1007/BF03393163

CrossRef Full Text | Google Scholar

Binder, C. (2003). Doesn't Everybody Need Fluency? Perf. Improv. 42 (3), 14–20. doi:10.1002/pfi.4930420304

CrossRef Full Text | Google Scholar

Binder, C., Haughton, E., and Bateman, B. (2002). Fluency: Achieving True Mastery in the Learning Process. Available at: https://www.researchgate.net/profile/Ogholgol_Nazari/publication/269793677_On_Situating_the_Stance_of_Practice_in_SLA/links/56b44f9808ae61c480581e9d.pdf (Accessed June 26, 2020).

Google Scholar

Binder, C., and Watkins, C. L. (1990). Precision Teaching and Direct Instruction: Measurably superior Instructional Technology in Schools. Perform. Improvement Q. 3 (4), 74–96. doi:10.1111/j.1937-8327.1990.tb00478.x

CrossRef Full Text | Google Scholar

Bolich, B., and Sweeney, W. J. (1996). An Eleven-Year Old Girl’s Use of Repeated reading SAFMEDS and See/write-Think/write Practice to Develop Fluent reading in Hebrew. J. Precision Teach. Celeration 14 (1), 57–71. Availableat: https://celeration.org/wp-content/uploads/2020/05/1996_JPTC_V14.01_07.pdf

Google Scholar

Braun, V., and Clarke, V. (2013). Successful Qualitative Research: A Practical Guide for Beginners. Croydon, United Kingdom: Sage.

Braun, V., and Clarke, V. (2012). “Thematic Analysis,” in APA Handbooks in Psychology®. APA Handbook of Research Methods in Psychology. Research designs: Quantitative, qualitative, neuropsychological, and biological. Editors H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, and K. J. Sher (Washington, DC: American Psychological Association), 2, 57–71. doi:10.1037/13620-004

CrossRef Full Text | Google Scholar

Braun, V., and Clarke, V. (2006). Using Thematic Analysis in Psychology. Qual. Res. Psychol. 3 (2), 77–101. doi:10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar

Calkin, A. B. (2005). Precision Teaching: The Standard Celeration Charts. Behav. Analyst Today 6 (4), 207–215. doi:10.1037/h0100073

CrossRef Full Text | Google Scholar

Casey, J., McLaughlin, T. F., Weber, K. P., and Everson, M. (2003). The Effects of Five-Minute Practice, Unlimited Practice, with SAFMED Cards on Correct and Error Rate in Math Facts for Two Elementary School Children with Learning Disabilities. Int. J. Spec. Education 18 (1), 66–72. 10.1.1.627.1342

Google Scholar

Cihon, T. M., Sturtz, A. M., and Eshleman, J. (2012). The Effects of Instructor-Provided or Student-Created Flashcards with Weekly, One-Minute Timings on Unit Quiz Scores in Introduction to Applied Behavior Analysis Courses. Eur. J. Behav. Anal. 13, 47–57. doi:10.1080/10521149.2012.11434404

CrossRef Full Text | Google Scholar

Common, E. A., and Lane, K. L. (2017). “Social Validity Assessment,” in Applied Behavior Analysis Advanced Guidebook: A Manual for Professional Practice. Editors J. K. Luiselli (Kansas: Elsevier: Academic Press), 73–92. doi:10.1016/b978-0-12-811122-2.00004-8

CrossRef Full Text | Google Scholar

Cunningham, D., McLaughlin, T. F., and Weber, K. P. (2012). The Use and Evaluation of Verbal Prompting with See to Say Problems and Answers and SAFMEDS to Teach Math Facts to a Student with Learning Disabilities. Int. J. Adv. Psychol. 1 (2), 37–39. Retried from: https://ia800602.us.archive.org/19/items/IJAP007/IJAP007.pdf

Google Scholar

Dalton, D. W., and Hannafin, M. J. (1988). The Effects of Computer-Assisted and Traditional Mastery Methods on Computation Accuracy and Attitudes. J. Educ. Res. 82 (1), 27–33. doi:10.1080/00220671.1988.10885861

CrossRef Full Text | Google Scholar

Durlak, J. A., and DuPre, E. P. (2008). Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. Am. J. Community Psychol. 41 (3-4), 327–350. doi:10.1007/s10464-008-9165-0

CrossRef Full Text | Google Scholar

Eshleman, J. W. (1985). Improvement Pictures with Low Celerations: An Early Foray into the Use of SAFMEDS. J. Precision Teach. 6 (3), 54–63. Retried from: https://celeration.org/wp-content/uploads/2017/02/JPTC_V06.03_02.pdf

Google Scholar

Foster, S. L., and Mash, E. J. (1999). Assessing Social Validity in Clinical Treatment Research: Issues and Procedures. J. Consulting Clin. Psychol. 67 (3), 308–319. doi:10.1037/0022-006x.67.3.308

CrossRef Full Text | Google Scholar

Fuller, J. L., and Fienup, D. M. (2018). A Preliminary Analysis of Mastery Criterion Level: Effects on Response Maintenance. Behav. Anal. Pract. 11 (1), 1–8. doi:10.1007/s40617-017-0201-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Glasgow, R. E., Lichtenstein, E., and Marcus, A. C. (2003). Why Don't We See More Translation of Health Promotion Research to Practice? Rethinking the Efficacy-To-Effectiveness Transition. Am. J. Public Health 93 (8), 1261–1267. doi:10.2105/AJPH.93.8.1261

CrossRef Full Text | Google Scholar

Graf, S. A., and Auman, J. (2005). SAFMEDS: A Tool to Build Fluency. Youngstown, OH: Graf Implements.

Greene, I., Tiernan, A. M., and Holloway, J. (2018). Cross Age Peer Tutoring and Fluency-Based Instruction to Achieve Fluency with Mathematics Computation Skills: A Randomized Controlled Trial. J. Behav. Educ. 27 (2), 145–171. doi:10.1007/s10864-018-9291-1

CrossRef Full Text | Google Scholar

Hunter, S. H., Beverley, M., Parkinson, J., and Hughes, J. C. (2016). Increasing High School Students' Maths Skills with the Use of SAFMEDS Class-wide. Eur. J. Behav. Anal. 17 (2), 154–165. doi:10.1080/15021149.2016.1247643

CrossRef Full Text | Google Scholar

Hutchinson, S., and Wilson, H. S. (1992). Validity Threats in Scheduled Semistructured Research Interviews. Nurs. Res. 41 (2), 117–119. doi:10.1097/00006199-199203000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, K. R., and Layng, T. V. J. (1996). On Terms and Procedures: Fluency. Behav. Analyst 19 (2), 281–288. doi:10.1007/BF03393170

CrossRef Full Text | Google Scholar

Johnson, K., and Street, E. M. (2012). Response to Intervention and Precision Teaching: Creating Synergy in the Classroom. New York: Guilford Press.

Kortesluoma, R. L., Hentinen, M., and Nikkonen, M. (2003). Conducting a Qualitative Child Interview: Methodological Considerations. J. Adv. Nurs. 42 (5), 434–441. doi:10.1046/j.1365-2648.2003.02643.x

CrossRef Full Text | Google Scholar

Kubina, R. M., and Yurich, K. K. L. (2012). The Precision Teaching Book. Lemont, PA: Greatness Achieved.

Lindsley, O. R. (1990). Precision Teaching: By Teachers for Children. TEACHING Exceptional Child. 22 (2), 10–15. doi:10.1177/004005999002200302

CrossRef Full Text | Google Scholar

Lindsley, O. R. (1996). The Four Free-Operant Freedoms. Behav. Analyst 19 (2), 199–210. doi:10.1007/BF03393164

CrossRef Full Text | Google Scholar

Meindl, J. N., Ivy, J. W., Miller, N., Neef, N. A., and Williamson, R. L. (2013). An Examination of Stimulus Control in Fluency-Based Strategies: SAFMEDS and Generalization. J. Behav. Educ. 22 (3), 229–252. doi:10.1007/s10864-013-9172-6

CrossRef Full Text | Google Scholar

Merbitz, C., Vieitez, D., Merbitz, N. H., and Pennypacker, H. S. (2004). “Precision Teaching,” in Evidence-Based Educational Methods. Editors D. J. Moran, and R. W. Malott (San Diego, California: Elsevier Academic Press), 47–62. doi:10.1016/b978-012506041-7/50005-x

CrossRef Full Text | Google Scholar

Nam, S. S., and Spruill, M. (2005). Learning Channel Intervention to Develop and Generalize Fluency in Multiplication Facts. J. Early Intensive Behav. Intervention 2 (2), 103–111. doi:10.1037/h0100305

CrossRef Full Text | Google Scholar

Nelson, J., and Campbell, C. (2017). Evidence-informed Practice in Education: Meanings and Applications. Educ. Res. 59 (2), 127–135. doi:10.1080/00131881.2017.1314115

CrossRef Full Text | Google Scholar

Owen, K. L., Hunter, S. H., Watkins, R. C., Payne, J. S., Bailey, T., Gray, C., et al. (2021). Implementation Support Improves Outcomes of a Fluency-Based Mathematics Strategy: A Cluster-Randomized Controlled Trial. J. Res. Educ. Effectiveness, 1–20. accepted/in press. doi:10.1080/19345747.2021.1875526

CrossRef Full Text | Google Scholar

Owen, K. (2020). Using Evidence-Based Methods to Support Children ‘at Risk’ of Poor Academic Outcomes to Develop Their Numeracy skills, Doctoral Thesis. Bangor, Gwynedd: Bangor UniversityRetried from: https://research.bangor.ac.uk/portal/files/36209501/Owen_K_PhD_thesis_Methods_to_support_mathematical_development.pdf (Accessed March 16, 2021).

Google Scholar

Patton, M. Q. (1999). Enhancing the Quality and Credibility of Qualitative Analysis. Health Serv. Res. 35 (5), 1189–1208.

Google Scholar

Potts, L., Eshleman, J. W., and Cooper, J. O. (1993). Ogden R. Lindsley and the Historical Development of Precision Teaching. Behav. Analyst 16, 177–189. doi:10.1007/BF03392622

CrossRef Full Text | Google Scholar

Quigley, S. P., Peterson, S. M., Frieder, J. E., and Peck, K. M. (2018). A Review of SAFMEDS: Evidence for Procedures, Outcomes and Directions for Future Research. Perspect. Behav. Sci. 41, 283–301. doi:10.1007/s40614-017-0087-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwartz, I. S., and Baer, D. M. (1991). Social Validity Assessments: Is Current Practice State of the Art? J. Appl. Behav. Anal. 24, 189–204. doi:10.1901/jaba.1991.24-189

CrossRef Full Text | Google Scholar

Stockwell, F., and Eshlelman, J. (2010). A Case Study Using SAFMEDS to Promote Fluency with Skinner’s Verbal Behavior Terms. J. Precision Teach. Celeration 26, 33–40.

Google Scholar

Thornicroft, G. H., Lempp, H., and Tansella, M. (2011). The Place of Implementation Science in the Translational Medicine Continuum. Psychol. Med. 41 (11), 2015–2021. doi:10.1017/s0033291711000109

PubMed Abstract | CrossRef Full Text | Google Scholar

Tolich, M., and Davidson, C. (2003). “Collecting the Data,” in Social Science Research in New Zealand. Editors C. Davidson, and M. Tolich. 2nd ed. (Auckland: Pearson Education), 121–153.

Google Scholar

Tuval-Mashiach, R. (2017). Raising the Curtain: The Importance of Transparency in Qualitative Research. Qual. Psychol. 4 (2), 126–138. doi:10.1037/QUP0000062

CrossRef Full Text | Google Scholar

Van Selm, M., and Jankowski, N. W. (2006). Conducting Online Surveys. Qual. Quant 40 (3), 435–456. doi:10.1007/s11135.005.8081.8

CrossRef Full Text | Google Scholar

Vargas, J. S. (2013). Behavior Analysis for Effective Teaching. New York: Routledge. doi:10.4324/9780203119303

CrossRef Full Text

Westfall, J. M., Mold, J., and Fagnan, L. (2007). Practice-Based Research-"Blue Highways" on the NIH Roadmap. Jama 297 (4), 403–406. doi:10.1001/jama.297.4.403

PubMed Abstract | CrossRef Full Text | Google Scholar

White, O. R., and Neely, M. D. (2012). The Chart Book: An Overview of Standard Celeration Chart Conventions and Practices. Retrieved from: https://education.uw.edu/sites/default/files/areas/edspe/white/readings/chartbook.pdf (Accessed June 3, 2021).

Google Scholar

White, O. R. (1986). Precision Teaching-Precision Learning. Exceptional Child. 52 (6), 522–534. doi:10.1177/001440298605200605

CrossRef Full Text | Google Scholar

Willig, C. (2008). Introducing Qualitative Research in Psychology. Berkshire: McGraw-Hill Education. doi:10.4135/9781848607927

CrossRef Full Text

Wyse, D. (2001). Promising Yourself to Do Better?: Target-Setting and Literacy. Education 3-13 29 (2), 13–18. doi:10.1080/03004270185200161

CrossRef Full Text | Google Scholar