Design, Validation, and Reliability of an Observation Instrument for Technical and Tactical Actions of the Offense Phase in Soccer

The use of observational methodology in the sports context provides coaches and other sports professionals with flexible tools that adapt to their needs. In collective sports, the use of these instruments is common for the technical and tactical analysis of the game. Based on the importance of data quality in these instruments, the purpose was to design, validate, and test the reliability of a mixed observational instrument of field formats and category systems to analyze technical and tactical actions in the offense phase in soccer. The instrument collects information regarding the actions with the ball, moment of the play (start, development, and end), and contextual situation for the offensive team and for the goalkeeper. The instrument design, validation, and reliability calculation were done in four stages: (a) review of the literature, (b) design the first draft of the instrument, (c) experts’ qualitative and quantitative review of the instrument, and (d) observer training test (reliability calculation). The content validity was established by 12 experts (Ph.D. in sports science or soccer coach with at least of 10 years of coaching experience). The Delphi methodology was used. Experts did a quantitative (scale 0–10) and qualitative evaluation. Experts were asked about: (a) comprehension of the criteria, categorical cores, degree of openness, and their definitions, (b) pertinence of categorical cores and degree of openness, and (c) whether to include other categorical cores or degree of openness in the observation instrument. The lowest Aiken’s V index was 0.91 for the categorical core “numerical situation with opponent goalkeeper.” The inter- and intra-observer reliability presented good levels of agreement. The lowest Kappa index was 0.96 for the inter-reliability in the categorical core “defensive pressing lines” and was 0.98 for the intra-reliability in the categorical core “ball height (start of ball possession),” “distance of the defensive player,” “ball height (end of ball possession),” “numerical situation,” and “defensive pressing lines.” The coefficients of the generalizability analysis showed a high level of accuracy, validity and reliability of the instrument. The results show that the instrument allows to obtain objective, valid and reliable information about the offensive phase in soccer.

Introduction

The study of the actions done by the players and team (i.e., match analysis) is common in soccer (Carling et al., 2006). The goal of team analysis is to give coaches useful information about their players, team, and opponents in order to design training and to prepare their matches (Hughes and Franks, 2004). From a research perspective, the analysis of game has focused on finding patterns in the game or performance indicators (Hughes, 2003; Dufour et al., 2017). For this purpose, the observational methodology is used to collect information in team sports, as it allows for the collection of multiple variables that interact in the sporting context (Anguera, 2003; Anguera and Hernández-Mendo, 2015; Fabra et al., 2018; Maneiro et al., 2018; Nadal et al., 2018). In this sense, observational instruments are used to collect information about the behaviors done by players and teams in order to register indicators to improve sports performance (Anguera et al., 2017, 2018; Maneiro et al., 2017; Moreno and Gómez-Ruano, 2017; Morillo et al., 2017; Serna Bardavío et al., 2017; Chacón-Moscoso et al., 2018). The rules and characteristics of the game of soccer, such other field sports, made that these behaviors are complex in nature, uncertainty, and multifactorial (Vilar et al., 2012). For that reason, most of the observational instruments are created ad hoc to solve the needs of the coaches or to answer a specific research problem. However, there is also possible to find in the literature generic and specific observational instruments, such the GAIP-Soccer, TSAP, SOF-5, SOFBAS, FUT-SAT or SoccerEye. These instruments have similarities in the analysis of the players and team behaviors, such technical actions and spatial and temporal analysis. However, most of these instruments are just focused on the field players’ actions and they do not integrate the goalkeeper actions and the contextual situations of the players’ actions with the ball. The current development of football requires that coaches, researchers and performance analysts use instruments that brings together all aspects of the game. These instruments are required not only to integrate the technical actions of the outfield players, but also to analyze aspects such as the influence of the goalkeeper on the offensive game, the numerical situations that occur and the incidence of the outfield area on the technical–tactical actions that occur.

Match analysis has been part of soccer for a several decades (Hughes and Franks, 2004). The introduction of technology, especially specialized software and video, has increased the used of players’ and team analysis by coaches and researchers. The first observation instruments were linked to specific software’s [e.g., Noldus or Sportcode (Hughes, 2003)]. The analysis of data quality is one of the bases on which the observational methodology is based. This is critical to collect precise, objective and reliable data about the players’ actions. In the 1990s, several generic observation instruments were designed and validated. Examples of these instruments are the Team Sports Assessment Procedure (Gréhaigne et al., 1997) and Game Performance Assessment Instrument (Oslin et al., 1998). These instruments were generic, but they can be adapted easily to the different team sports (Harvey et al., 2010). In the two last decades, several specific observation instruments have developed in soccer. An incomplete list of these instruments is: (a) SOF (currently in its fifth version), initially developed by Anguera et al. (2003); (b) SOFBAS (currently in its second version) initially developed by Castellano (2000), (c) FUT-SAT, developed by Da Costa et al. (2011); and (d) SoccerEye, developed by Barreira et al. (2012). There is also possible to find in the literature a large number of observation instruments generate ad hoc to solve specific research problems, such as set-plays, activity profiles and group behavior (Sarmento et al., 2017). The analysis of these instruments and the proposal of categorical cores found in the reviewed literature show similar patterns. They collect information related the start of the offensive phase, the progress of the ball possession (attack and defense player/team), and the end of the offensive phase. The information collected about these phases described the actions done by the players and the situation in which these actions are done (e.g., temporal and spatial patterns). Most of these instruments are focused on the study of the field players. Only studies focus on the specific analysis of the goalkeeper had analyzed the actions of this player (Sainz de Baranda et al., 2005; Sainz de Baranda et al., 2008; Abellán et al., 2017).

Reviewed instruments analyze isolated actions, not integrated in the action in their context or players involved. In the same way, the analysis of the game of soccer requires a valid and reliable instrument. The dynamics of the game needs an instrument that analyzes the continuity of the player’s actions with the ball, without isolating the actions of the game. The current tendencies of the game require the integration of the goalkeeper as a critical element that participated on the team’s offense and the defense. The dynamics of the game need an instrument that analyses the continuity of the game, without isolating game actions. In addition, the goalkeeper’s offensive actions and numerical situations need to be integrated, as his participation in offensive actions in today’s football has increased (Florin, 2009; Szwarc et al., 2010). The proposal of observation instrument developed in this paper tries to integrate categorical cores from previous research and instruments and to integrate the goalkeeper in the analysis of the game players’ and team patterns. This observational instrument will provide information about how the technical and tactical actions are realized and will help to provide more information related to the dynamic system that involves team sport confrontation (Glazier, 2010; Castellano et al., 2017; González-Espinosa et al., 2017). The purpose of this research was to design, validate, and test the reliability of an observation instrument to analyze the offensive technical and tactical individual ball actions in soccer.

Materials and Methods

Design

For the development of the instrument an observational, nomothetic, monitoring and multidimensional design was carried out (Anguera and Hernández-Mendo, 2013). The observation instrument designed was a mixed field format and category system (Anguera and Hernández-Mendo, 2013). The final instrument was composed of 3 criteria and 25 categorical cores. The multifaceted design for the generalizability analysis was composed of three facets: [Observers], [criteria], and [categorical cores]. About these facets three designs were analyzed: [Criteria] [Categorical cores]/[Observers], [Observers]/[Criteria] [Categorical cores] and [Observers] [Criteria]/[Categorical cores].

Participants

The sample studied consisted of 12 matches played by 44 players belonging to four U-12 teams during three tournaments played after the end of the regular season.

Instrument

The dimensions of the criteria collected by the instrument were divided into three groups: the first criterion, start of the ball possession; the second criterion, development (technical–tactical actions done with the ball), and the third criterion, the end of the ball possession. The unit of analysis was the play phase with the ball. The established categories were exhaustive and mutually exclusive (E/ME) (Anguera and Hernández-Mendo, 2013).

Researchers did a pilot study of the observational instrument with under-12 matches to review and complete the categorical cores and degree of openness. The list of categorical cores was grouped into three groups of criteria: (a) start of the ball possession, (b) development (technical–tactical actions done with the ball), and (c) end of the ball possession. Ten categorical cores described the criterion “start of the ball possession” (way of obtaining the ball, ball height, body part, origin zone, zone where the ball was controlled (see Figures 1, 2), numerical situation (offense players vs. defense players), numerical situation with opponent goalkeeper (Offense players vs. Defense players + Goalkeeper), numerical situation with own goalkeeper (Goalkeeper with ball + Offense players vs. Defense players), distance of the defensive player, and teammate support). Five categorical cores described the criterion “development of the ball possession” (tactical collective actions, dribble, ball touches, type of ball contact, and defensive pressing lines). Twelve categorical cores described the criterion “end of the ball possession” [technical action, body part, height, zone where ball possession ends (see Figures 1, 2), goalkeeper zone intervention, zone where the ball ends (see Figures 1, 2), numerical situation (offense players vs. defense players), numerical situation with opponent goalkeeper (Offense players vs. Defense players + Goalkeeper), numerical situation with own goalkeeper (Goalkeeper with ball + Offense players vs. Defense players), and teammate support]. The definition of all the categorical cores and degrees of openness can be found in Annexes 1–3 of the present paper.

FIGURE 1

Figure 1. Division of the field used to establish the zone from which the ball was sent.

FIGURE 2

Figure 2. Division of the goal area used to establish the zone where the goalkeeper handled the ball.

Procedure

The instrument design, validation, and reliability calculation were done in four stages: (a) review of the literature, (b) design the first draft of the observation instrument, (c) experts’ qualitative and quantitative review of the instrument, and (d) observer training test (reliability calculation). In the first stage, a review of the following databases was done: Web of Science (WOS) de ISI Thomson Reuters, Latindex, Sports Discus, Scopus, Google Scholar, Scielo, and Dialnet. The keywords of the search were: “soccer” or “football” and “observational instrument.” A review of the abstracts was carried out to select the observational instruments used in the literature. After the selection of papers with observational instruments, the researchers reviewed their characteristics, criteria and categorical cores. In the second stage, a draft of a list of categorical cores and degree of openness was created from related scientific literature. The list of criteria included the categories, degree of openness and the behavior’s definition.

In the third stage, content validity was established by 12 experts (Ph.D. in sports science or soccer coach with at least of 10 years of coaching experience). The Delphi methodology was used. Experts did a quantitative (scale 0–10) and qualitative evaluation. Experts were asked about: (a) comprehension of the criteria, categorical cores, degree of openness, and their definitions, (b) pertinence of categorical cores and degree of openness, and (c) whether to include other categorical cores or degree of openness in the observation instrument. The content validity was calculated with the Aikens’s V coefficient (Aiken, 1985). A Visual Basic app was used to calculate it (Merino and Livia, 2009). The confidence intervals were set out in 90%, 95%, and 99% through score method (Penfield and Giacobbi, 2004). Items with average values <0.70 were eliminated, items ≥0.70 and <0.8 were reviewed according to experts’ proposals, and items that were ≥8.0 were accepted (Bulger and Housner, 2007).

In the fourth stage, the reliability of the instrument was calculated through an observer training and an observation test. Two observers were trained in the use of the observation instrument during three 2-h sessions. The observers had degrees in sports science, had 10 years of experience coaching soccer, and had experience as observers. After the training, a match analysis was carried out by the observers and an expert observer (Master in Sports Science with 5-years of experience in match analysis and soccer). After minimum 15 days, the same analysis was carried out again by the two observers (observer and researcher, Ph.D. in Sports Science with 15-years of experience in match analysis and coaching). The inter- and intra-observer Cohen’s kappa, the intra-class correlation coefficient and Kendall’s Tau B was utilized to evaluate observer agreement. A generalizability analysis (Tables 6, 7) was performed to test the validity and accuracy of the observation instrument and the reliability of the observers (Blanco-Villaseñor et al., 2000; Morillo et al., 2017). For the statistical analysis the SPSS software version 24.0 was used. For the generalizability analysis, SGAT software was used (Hernández-Mendo et al., 2016).

Results

The list of categorical cores, degree of openness and their definitions after the first and second stages of the observational instrument design is shown in Tables 1–3. In the design of the observational instrument, the categorical cores and degree of openness were selected using the categorical cores and degree of openness proposed by Moreno (2005), Sainz de Baranda et al. (2005), Anguera et al. (2011), García-López et al. (2013), Hernández-Mendo et al. (2014), and Anguera and Hernández-Mendo (2015).

TABLE 1

Table 1. Categorical cores, degree of openness, and definitions related to the criterion “start of the ball possession.”

TABLE 2

Table 2. Categorical cores, degree of openness, and definitions related to the criterion “development of the ball possession.”

TABLE 3

Table 3. Categorical cores, degree of openness, and definitions related to the criterion “end of the ball possession.”

In the third stage, the experts reviewed the observation instrument. The experts’ qualitative observations were related to the definitions of categorical cores. No categorical cores were eliminated by the experts after the experts’ evaluation. In the quantitative evaluation, all categorical cores had an average score >0.70 (Table 4). The lower Aiken’s V value found was 0.91, concretely in the categorical core “numerical situation with opponent goalkeeper.” In the fourth stage (observer training), the lowest intra and inter observer agreement coefficient observers was >0.96 for all the studied categorical cores (Cohen’s Kappa and interclass correlation coefficient); and the lowest intra and inter observer agreement coefficient observers was >0.97 for all the studied categorical cores (Kendall’s Tau B) (Table 5).

TABLE 4

Table 4. Values of content validity (Aiken’s V).

TABLE 5

Table 5. Observers’ intra and inter agreement after training in the use of the observation instrument (fourth stage).

Finally, the analysis of generalizability (Tables 6, 7) shows in the first design a Generalizability coefficient (GC) equal to 1, this result shows a high reliability of the observers. In the second and third design, the results of the GC are equal to or very close to 0, these results indicate a high adjustment of the observation instrument and that its categories were exhaustive and mutually exclusive (E/ME) (Anguera and Hernández-Mendo, 2013). The highest percentage of variance (see Table 7) is found in the interaction [Criteria] [Categorical cores], being very low in the rest of the sources of variation.

TABLE 6

Table 6. Absolute generalizability coefficient, relative generalizability coefficient, absolute standard deviation, and relative standard deviation in each of the designs.

TABLE 7

Table 7. Sources of variation, sum of squares, degrees of freedom, mean squares, % and standard error.

Discussion

The present paper describes the stages done to design, validate, and test the reliability and generalizability of the observational instrument. Considering the increasing number of observational studies that use “Ad hoc” instruments in football, the development of this instrument aims to provide the many observational studies that occur in football with a valid and reliable instrument that allows adequate data collection. In the same way, the developed instrument has the advantage of using open categories, as opposed to other studies that present closed categories, in which only the conducts carried out are registered. In addition, the designed instrument has the advantage that it takes into account the continuity of the game. The instrument registers the actions that happens since the players obtain ball possession until they lose it.

The fact of being able to record the continuity of the offensive game, the procedures by which it is made with the ball, the behaviors it manifests in possession of the ball and finally, the actions developed when it is released from the ball, is an advantage over other instruments that analyze and record actions in an isolated way, for example instruments that analyze goals (Reina-Gómez et al., 2010). In addition this instrument provides, in spite of analyzing the continuity of the offensive action, do not take into account categorical cores of relevance as the actions of the goalkeeper, the corporal zones used in the action, the existence of companions in support or the defensive lines that are surpassed in the offensive action (Sarmento et al., 2010).

The process had different stages, similar to the one followed in the development of observational instruments in other sports (Villarejo et al., 2014; Palao et al., 2015a,b). The first stage involved the review and analysis of the available observation instruments and match analysis literature. The analysis of the available material was focused in the design an instrument that allows obtaining information: (a) about the individual and collective game patterns, (b) about the players evolution in training and competition, (c) about the effect of manipulation of rules in small side-games or in competition, and (d) that guide the training design. After the analysis, the researchers established the categorical cores focus on the following aspects: the sequence of the ball possession (stat, develop, and end), description of the actions done by the player with the ball, description of the temporal, spatial, and context of the actions done with the ball, and inclusion of the goalkeeper in the analysis of the situation. The researchers tried to use in the design of the instrument categorical cores close to the criteria used by coaches with the goal to increase the applicability of the instrument. For that reason, the criteria used for the Spanish Soccer Federation to describe the technical and tactical actions with the ball in the coach training courses was followed (Moreno, 2005). Along with this criterion, it was also used the categorical cores proposed by researchers in other instruments and research studies (Sainz de Baranda et al., 2005; Anguera et al., 2011; García-López et al., 2013; Hernández-Mendo et al., 2014; Anguera and Hernández-Mendo, 2015). The instrument has as unit of analysis the ball possession and differentiates in the ball possession three moments (start, development of end), like other instruments in soccer (e.g., SOFBAS or SoccerEye). The pilot studied carried out by the researchers allowed reviewing the categorical cores and their definitions.

In the third stage of the process, the experts’ qualitative and quantitative revision and the use of Aiken’s V allowed for measuring the content validity of the items. The values of the quantitative evaluation were high for all the categorical cores (Vo ≥ 0.70) and above the minimum values proposed in the literature as a reference (Vo = 0.70) by Penfield and Giacobbi (2004). The qualitative evaluation helped specifically to clarify some of the aspects of the definitions of the different degree of openness of the categorical cores. The use of experts from research and coaching provided a more complete and holistic vision of the sport.

The level of intra-reliability, reached between observers after the observer training, showed the instrument is adequate with regard to reliability (Bakeman et al., 1997). The use of categorical cores and degree of openness well established in the area and the qualitative evaluation of the experts contributed to the high level of reliability. Finally, the results shown in the generalizability analysis made it possible to verify the high levels of validity, precision of the instrument and corroborate the high reliability indices of the observers (Blanco-Villaseñor et al., 2000; Hernández-Mendo et al., 2016).

Data obtained by the instrument can provide useful and applicable information to coaches in order to understand game patterns, face the competition and develop the training sessions. The structure of the instrument allows coaches and researchers to use it in its entirety or certain parts or criteria can be utilized to solve specific research problems.

Conclusion

The results showed that the observational instrument is valid and reliable for measuring the technical and tactical actions done in the offensive phase by the player and team with the ball possession. The instrument has some limitations. Only it was assessed the content validity of the instrument (expert evaluation), and the instrument is focused on the actions of the players and team which the ball possession. However, the instrument can provide coaches and research with information about the type of actions done by the players with the ball, their characteristics and in which context are done (distance, numerical situation) and the level of involvement of the field players and the goalkeeper. This information could allow coaches and research to establish the demands of the game and to create training plans that help players to be prepared for these demands.

Ethics Statement

This study respected the ethical principles established by the UNESCO Declaration on Bioethics and Human Rights. The parents or guardians of the players were informed of the study and gave their written consent in accordance with the Declaration of Helsinki. The study was approved by the Ethics Committee of University of Murcia (Spain) with ID 1944/2018.

Author Contributions

All authors participated in the design, documentation, development, and writing of the manuscript. This paper was reviewed by all authors and all of them are responsible for its contents and providing they are responsible for the final version.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article. This work was supported by the Ministry of Science and Innovation, Spain, under grant DEP2010-16140 and Project “Salvador de Madariaga”, of the Ministry of Education, Culture and Sport (PRX16/00422). This study is part of the project entitled “Estudio del Riesgo de Lesión en Jóvenes Deportistas a través de Redes de Inteligencia Artificial”, funded by Spanish Ministry of Science and Innovation (DEP2017-88775-P), the State Research Agency (AEI) and the European Regional Development Fund (ERDF).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00022/full#supplementary-material

References

Abellán, J., Sáez-Gallego, N. M., Vila-Maldonado, S., and Contreras, O. (2017). Differences in movement behaviour between successful and less successful goalkeepers in the interception of corner kicks. J Hum. Sport Excerc. 12, 307–315. doi: 10.14198/jhse.2017.122.07

CrossRef Full Text | Google Scholar

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educ. Psychol. Meas. 45, 131–142. doi: 10.1177/0013164485451012

PubMed Abstract | CrossRef Full Text | Google Scholar

Anguera, M. T. (2003). “Observational methods,” in Encyclopedia of Psychological Assessment, ed. R. Fernández-Ballesteros (London: Sage),632–637.

Google Scholar

Anguera, M. T., Blanco-Villaseñor, A., Hernández-Mendo, A., and Losada, L. (2011). Observational designs: their suitability and application in sports psychology. Cuad. Psicol. Dep. 11, 63–76.

Google Scholar

Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., Ardá, T., Camerino, O., Castellano, J., et al. (2003). Instrumento de codificación y registro de la acción de juego en fútbol – SOF-1 (Instrument for coding and recording football action - SOF-1). Rev. Digit. Alto Rendimiento Fútbol [electronic version].

Anguera, M. T., Camerino, O., Castañer, M., Sánchez-Algarra, P., and Onwuegbuzie, A. J. (2017). The specificity of observational studies in physical activity and sports sciences: moving forward in mixed methods research and proposals for achieving quantitative and qualitative symmetry. Front. Psychol. 8:2196. doi: 10.3389/fpsyg.2017.02196

PubMed Abstract | CrossRef Full Text | Google Scholar

Anguera, M. T., and Hernández-Mendo, A. (2013). Observational methodology in sport science. EBM. Com. Rev. Cienc. Dep. 9, 135–160.

Anguera, M. T., and Hernández-Mendo, A. (2015). Data analysis techniques in observational studies in sport sciences. Cuad. Psicol. Dep. 15, 13–29. doi: 10.4321/S1578-84232015000100002

CrossRef Full Text | Google Scholar

Anguera, M. T., Portell, M., Chacón-Moscoso, S., and Sanduvete-Chaves, S. (2018). Indirect observation in everyday contexts: concepts and methodological guidelines within a mixed methods framework. Front. Psychol. 9:13. doi: 10.3389/fpsyg.2018.00013

PubMed Abstract | CrossRef Full Text | Google Scholar

Bakeman, R., Quera, V., McArthur, D., and Robinson, B. F. (1997). Detecting sequential patterns and determining their reliability with fallible observers. Psychol. Methods 2, 357–370. doi: 10.1037/1082-989X.2.4.357

CrossRef Full Text | Google Scholar

Barreira, D., Garganta, J., Prudente, J., and Anguera, M. T. (2012). Desenvolvimento e validação de um sistema de observação aplicado à fase ofensiva em Futebol: Soccer Eye. Rev. Port. Ciênc. Desporto 12, 32–57. doi: 10.5628/rpcd.12.03.32

PubMed Abstract | CrossRef Full Text | Google Scholar

Blanco-Villaseñor, A., Castellano, J., and Hernández-Mendo, A. (2000). Generalizabilidad de las observaciones en la acción del juego en el fútbol. Psicothema 12, 81–86.

Google Scholar

Bulger, S. M., and Housner, L. D. (2007). Modified Delphi investigation of exercise science in physical education teacher education. J. Teach. Phys. Educ. 26, 57–80. doi: 10.1123/jtpe.26.1.57

CrossRef Full Text | Google Scholar

Carling, C., Williams, A. M., and Reilly, T. (2006). Handbook of soccer match analysis: a systematic approach to improving performance. J. Sport Sci. Med. 5:171.

Google Scholar

Castellano, J. (2000). Observación y Análisis de la Acción de Juego en Fútbol. Ph.D. thesis, University of the Basque Country, Álava.

Castellano, J., Fernández, E., Echeazarra, I., Barreira, D., and Garganta, J. (2017). Influence of pitch length on inter-and intra-team behaviors in youth soccer. Ann. Psychol. 33, 486–496. doi: 10.6018/analesps.33.3.271051

CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Sanduvete-Chaves, S., Anguera, M. T., Losada, J. L., Portell, M., and Lozano-Lozano, J. A. (2018). Preliminary checklist for reporting observational studies in sports areas: content validity. Front. Psychol. 9:291. doi: 10.3389/fpsyg.2018.00291

PubMed Abstract | CrossRef Full Text | Google Scholar

Da Costa, I. T., Garganta, J., Greco, P. J., Mesquita, I., and Maia, J. (2011). System of tactical assessment in Soccer (FUT-SAT): development and preliminary validation. Motricidade 7, 69–84. doi: 10.6063/motricidade.7(1).121

CrossRef Full Text | Google Scholar

Dufour, M., Phillips, J., and Ernwein, V. (2017). What makes the difference? Analysis of the 2014 World Cup. J. Hum. Sport Excerc. 12, 616–629. doi: 10.14198/jhse.2017.123.06

CrossRef Full Text | Google Scholar

Fabra, P., Balaguer, I., Tomás, I., Smith, N., and Duda, J. L. (2018). Versión española del Sistema de Observación del Clima Motivacional Multidimensional (MMCOS): fiabilidad y evidencias de validez. Rev. Psicol. Dep. 27, 11–22.

Google Scholar

Florin, T. D. (2009). The identification of the value dimensions of the high performance goalkeeper pattern. Ovidius Univ. Ann. Ser. Phys. Educ. Sport Sci. Mov. Health 9, 82–85.

Google Scholar

García-López, L. M., González-Víllora, S., Gutiérrez-Díaz, D., and Serra, J. (2013). Development and validation of the game performance evaluation tool (GPET) in soccer. Sport Rev. Eur. Cienc. Dep. 1, 89–79.

Google Scholar

Glazier, P. S. (2010). Game, set and match? Substantive issues and future directions in performance analysis. Sports Med. 40, 625–634. doi: 10.2165/11534970-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

González-Espinosa, S., Molina, S. F., García-Rubio, J., Medina, A. A., and García-Santos, D. (2017). Diferencias en el aprendizaje según el método de enseñanza-aprendizaje en el baloncesto. Rev. Psicol. Dep. 26, 65–70.

Google Scholar

Gréhaigne, J. F., Godbout, P., and Bouthier, D. (1997). Performance assessment in team sports. J. Teach. Phys. Educ. 16, 500–516. doi: 10.1123/jtpe.16.4.500

CrossRef Full Text | Google Scholar

Harvey, S., Cushion, C. J., Wegis, H. M., and Massa-González, A. N. (2010). Teaching games for understanding in American high-school soccer: a quantitative data analysis using the game performance assessment instrument. Phys. Educ. Sport Pedagogy 15, 29–54. doi: 10.1080/17408980902729354

CrossRef Full Text | Google Scholar

Hernández-Mendo, A., Blanco-Villaseñor, A., Pastrana, J. L., Morales-Sánchez, V., and Ramos-Pérez, F. J. (2016). SAGT: aplicación informática para análisis de generalizabilidad [SAGT: new software for generalizability analysis]. Rev. Iberoam. Psicol. Ejerc. Dep. 11, 77–89.

Google Scholar

Hernández-Mendo, A., Castellano, J., Camerino, O., Jonsson, G., Blanco-Villaseñor, A., Lopes, A., et al. (2014). Observational software, data quality control and data analysis. Rev. Psicol. Dep. 23, 111–121.

Google Scholar

Hughes, M. (2003). “Notational analysis,” in Science and Socscer, eds T. Reilly and A. M. Williams (London: Routledge), 245–263.

Google Scholar

Hughes, M., and Franks, I. M. (2004). Notational Analysis of Sport: Systems for Better Coaching and Performance in Sport. London: Psychology Press.

Google Scholar

Maneiro, R., Amatria, M., Moral, J. E., and Lopez, A. (2018). Análisis observacional de las relaciones interlíneas de la Selección Española de Fútbol, mediante coordenadas polares. Cuad. Psicol. Dep. 18, 18–32.

Google Scholar

Maneiro, R., Losada, J. L., Casal, C. A., and Ardá, A. (2017). Multivariate analysis of indirect free kick in the FIFA World Cup 2014. Ann. Psychol. 33, 461–470. doi: 10.6018/analesps.33.3.271031

CrossRef Full Text | Google Scholar

Merino, C., and Livia, J. (2009). Confidence intervals for the content validity: a visual basic computer program for the Aiken’s V. Ann. Psychol. 25, 169–171.

Google Scholar

Moreno, E., and Gómez-Ruano, M. A. (2017). Validación herramienta observacional para el análisis de rachas de lanzamiento en baloncesto. Rev. Psicol. Dep. 26, 87–93.

Google Scholar

Moreno, M. (2005). Technical Education in Youth Soccer. Escuela Nacional de Entrenadores de Fútbol. Madrid: Gymnos.

Morillo, J. P., Reigal, R. E., Hernández-Mendo, A., Montaña, A., and Morales-Sánchez, V. (2017). Decision-making by handball referees: design of an ad hoc observation instrument and polar coordinate analysis. Front. Psychol. 8:1842. doi: 10.3389/psyg.2017.01842

PubMed Abstract | CrossRef Full Text | Google Scholar

Nadal, G., Serna, J., Nuviala, R., and Falcón, D. (2018). Diseño de un instrumento observacional para la valoración del penalti en fútbol y análisis de los resultados obtenidos. Rev. Psicol. Dep. 27, 189–199.

Oslin, J. L., Mitchell, S. A., and Griffin, L. L. (1998). The game performance assessment instrument (GPAI): development and preliminary validation. J. Teach. Phys. Educ. 17, 231–243. doi: 10.1123/jtpe.17.2.231

CrossRef Full Text | Google Scholar

Palao, J. M., Manzanares, P., and Ortega, E. (2015a). Design and validation of an observational instrument for technical and tactical actions in beach volleyball. Motriz 21, 137–147. doi: 10.5016/motriz.v21i2.8686

CrossRef Full Text | Google Scholar

Palao, J. M., Manzanares, P., and Ortega, E. (2015b). Design and validation of an observation instrument for technical and tactical actions in indoor volleyball. Eur. J. Hum. Mov. 34, 75–95.

Google Scholar

Penfield, R. D., and Giacobbi, P. R. (2004). Applying a score confidence interval to Aiken’s item content-relevance index. Meas. Phys. Educ. Exerc. Sci. 8, 213–225. doi: 10.1207/s15327841mpee0804_3

CrossRef Full Text | Google Scholar

Reina-Gómez, A., Hernández-Mendo, A., and Fernández-García, J. C. (2010). Multi-facet design for goal scoring in SOCCER-7. Qual. Quant. 44, 1025–1035. doi: 10.1007/s11135-009-9253-8

CrossRef Full Text | Google Scholar

Sainz de Baranda, P., Ortega, E., Llopis, L., Novo, J. F., and Rodríguez, D. (2005). Analysis of the goalkeeper’s defensive actions in football 7. Apunts 80, 45–52.

Google Scholar

Sainz de Baranda, P., Ortega, E., and Palao, J. M. (2008). Analysis of goalkeepers’ defence in the World Cup in Korea and Japan in 2002. Eur. J. Sport Sci. 8, 127–134. doi: 10.1080/17461390801919045

CrossRef Full Text | Google Scholar

Sarmento, H., Anguera, T., Campaniço, J., and Leitão, J. (2010). Development and validation of a notational system to study the offensive process in football. Medicine 46, 401–407.

PubMed Abstract | Google Scholar

Sarmento, H., Clemente, F. M., Araújo, D., Davids, K., McRobert, A., and Figueiredo, A. (2017). What performance analysts need to know about research trends in Association Football (2012–2016): a systematic review. Sports Med. 48, 1–38. doi: 10.1007/s40279-017-0836-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Serna Bardavío, J., Muñoz Arroyave, V., Hileno González, R., Solsona Leri, E., and Sáez de Ocáriz Granja, U. (2017). Patrones temporales iniciados con bloqueo directo o uno contra uno en baloncesto. Rev. Psicol. Dep. 26, 81–86.

Google Scholar

Szwarc, A., Lipinska, P., and Chamera, M. (2010). The efficiency model of goalkeeper’s actions in soccer. Balt. J. Health Phys. Act. 2, 132–138. doi: 10.2478/v10131-0013-x

CrossRef Full Text | Google Scholar

Vilar, L., Araújo, D., Davids, K., and Button, C. (2012). The role of ecological dynamics in analysing performance in team sports. Sports Med. 42, 1–10. doi: 10.2165/11596520-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

Villarejo, D., Ortega, E., Gómez, M. A., and Palao, J. M. (2014). Design, validation, and reliability of an observational instrument for ball possessions in rugby union. Int. J. Perform. Anal. Sport 14, 955–967. doi: 10.1080/24748668.2014.11868771

CrossRef Full Text | Google Scholar