Skip to main content

Evaluation of the Nursing Culture Assessment Tool for Pressure Injury Prevention: A Mixed-methods Study

Empirical Studies

Evaluation of the Nursing Culture Assessment Tool for Pressure Injury Prevention: A Mixed-methods Study

Index: Wound Management & Prevention 2019;65(12):32–40

Abstract

The nursing culture in long-term care (LTC) settings may affect quality measures such as pressure injury (PrI) rates. 

PURPOSE: The study was conducted to evaluate the relevance of an LTC facility’s nursing culture to both their quality measures and their staff’s perceptions of care in the context of PrI prevention. METHODS: Directors of Nursing (DONs) in 4 purposively selected Medicare/Medicaid-certified skilled nursing facilities were invited by phone, agreed to participate in the 5-day project, and completed an initial 7-item, facility-related survey. Their staff completed the Nursing Culture Assessment Tool (NCAT), a pen-and-paper instrument that comprises 19 items regarding 6 principal dimensions of nursing culture (behaviors, expectations, teamwork, communication, satisfaction, and professional commitment) and participated in focus groups to discuss the NCAT and its findings using standardized probes of the perception of survey salience in relation to PrI prevention practices. Staff, including registered nurses, licensed practical nurses, and certified nursing assistants employed either part- or full-time at each facility, were eligible for study participation over a 5-day period.  All data collection and analyses were conducted by the authors. Facility-related data were descriptive only. Analyses of variance were used to test differences in standardized NCAT scores by facility, and focus group transcripts were coded and subjected to structured thematic content analysis. RESULTS: One hundred, nine (109) people completed the NCAT, and 47 participated in focus groups. NCAT scores varied significantly by facility (P value range .001–.027). Staff comments about their respective facility’s results focused primarily on communication and teamwork and included both agreement or disagreement with the facility’s high or low scores in the context of PrI prevention, as well as suggestions for instrument administration. CONCLUSION: Examination of nursing culture using the NCAT can provide new and targeted perspectives on how frontline workers perceive barriers and facilitators to delivery of PrI prevention in LTC. To support the evidence base regarding their values and beliefs, future research on effective workplace change in LTC settings will require nuanced assessment of the meaning and impact of the nursing culture on worker performance.

Introduction

An organization’s culture may support or impede the effective discharge of its vision, mission, values, and systems. Based on research conducted in more than 50 long-term care (LTC) facilities in the United States, a negative health care work environment may jeopardize the quality of patient care, in part by impeding the effective recruitment and retention of qualified nursing staff.1,2 

Older adults living in LTC facilities often are frail and at risk of a range of undesirable outcomes, including pressure injury (PrI), the measure of which is regarded as a key clinical quality indicator of the care delivered by nursing. A PrI is any skin lesion, often over a bony prominence, resulting from the skin’s prolonged exposure to pressure that causes capillary occlusion and eventual tissue necrosis and cell death. Based on a representative sample of 114 380 LTC residents nationwide, the descriptive study by Redelings et al3 reported most PrIs are avoidable, and those that develop are associated with complications that compromise the resident’s health status and quality of life. LTC residence is, in itself, a risk factor for PrI development4; at least 1 of every 9 nursing home (NH) residents in the US experiences 1 PrI during their stay, and many LTC facilities have high PrI incidence and prevalence rates, in some instances more than 20%.5 The Centers for Medicare & Medicaid Services6 (CMS) report PrI prevalence rates in US LTC settings remain high, regardless of 30 years of evidence-based prevention guidelines and despite intervention. These data were supported by studies involving nearly 2000 older, racially diverse LTC residents and confirmed by systematic review of clinical trials in 5 databases.7,8 

Organizational culture creates the boundaries and guidelines that shape nursing staff behavior in LTC. Nursing culture consists not only of the organization’s core values, but also of discipline-specific values and beliefs that together create the norms that subsequently influence the delivery and quality of care LTC residents receive. Whereas the traditional assumption is that the organization’s core culture (comprised of all disciplines) drives care delivery, the current authors’ work suggests care norms, rituals, and other cultural dimensions embraced by the nursing staff are a unique subset of the core organizational culture and may extend beyond it in relation to programs for, and delivery of, clinical nursing care.1,2,9 A qualitative study10 of research and clinical staff (N = 10) suggested cultural patterns within a facility may interact with the characteristics of the staff and residents to degrade the work environment and the quality of patient care, a finding supported by methodological problems emerging from intervention studies in LTC. 11 These complex relationships also have been reported in a series of larger studies12  of nursing homes (N = 2840) representative of Texas LTC populations in institutions with leadership problems or high staff turnover rates.13,14  

The Nursing Culture Assessment Tool (NCAT) is a psychometrically valid instrument that provides a comprehensive assessment of nursing culture in a LTC facility.1 The NCAT’s 19 items capture 6 principal dimensions of nursing culture (behaviors, expectations, teamwork, communication, satisfaction, and professional commitment); the tool can be completed in less than 10 minutes.2 The NCAT has been shown to be a reliable and valid measure of organizational nursing culture in LTC with demonstrated dimensionality and construct validity; recent research proposed its potential contribution to planned quality improvement projects by targeting deficits in nursing culture,1 and the resulting impact on nursing subculture can promote or impede effective care delivery. This study extends that work by documenting whether and how nursing staff interpret NCAT results in the context of a specific quality improvement measure: the prevention of PrIs. By re-examining NCAT content validity in this context and exploring focus group perspectives on its accuracy in order to test whether nursing culture may be relevant to a specific targeted improvement in care, this study sought to examine how nursing staff rated and understood their own nursing culture in the context of a core clinical value in LTC: prevention of PrI. 

This study is the first to contextualize the NCAT by asking survey participants to think about a specific area of nursing practice such as PrI prevention practices in that LTC facility when responding to NCAT items. 

The research questions were:

1. How do the total nursing culture scale and its subscale scores reflect differences across LTC facilities in which the quality measures of PrI prevention differ?

2. How do the perspectives of a facility’s LTC nursing staff illuminate and explain its nursing culture scores in the context of PrI prevention?

Methods

Design and sample. The study employed an explanatory, sequential, mixed-method design. Eligible LTC facilities were identified from the publicly available list from the CMS; 57 Medicare/Medicaid-certified skilled nursing facilities/NHs that provided care services to older adults were located within 100 miles of Durham, NC. Of these facilities, 25 responded to a phone call from the researcher requesting participation; 16 declined, 5 responded “maybe,” and 4 agreed; these 4 NHs were designated A through D. The NHs varied by size (from 60 to 125 operational beds) and location (urban/suburban) in North Carolina. The principle investigator contacted the director of nursing (DON) at each facility to explain the study, invite participation, and set a mutually agreeable date for initiating the onsite activities of the 5-day project. Upon agreement, DONs completed a 7-item, facility-related survey before the intervention. 

All nursing staff (registered nurses [RNs], licensed practical nurses [LPNs], and certified nursing assistants [CNAs]) employed either part- or full-time at each NH were eligible for study participation. On day 1, nursing staff who were willing to participate and had the ability to read and understand English were given an envelope containing an introductory letter, a consent form, an 11-item demographic and work history survey, and the 19-item nursing culture survey (color coded for RN, LPN, and CNA job categories). No identifiers appeared on the tool other than facility number identifier and the paper color for job category. Participants were asked to review and sign the consent document, complete the NCAT survey, place both items in the envelope provided, and return the sealed envelope to a locked drop box at the nursing station within 3 days. On day 4, the NCAT survey responses were entered into a database, and the mean total score and mean subscale responses were summarized for presentation to nursing staff. 

On day 5, all participants were invited to a focus group conducted in each of the 4 facilities to elicit the nursing staff’s perspectives about the findings of the nursing culture assessment in their facility, to explore how well the results captured their nursing culture, and to determine whether and how they perceived nursing culture to be related to implementation of best practices for PrI prevention. 

Ethics approval for this study was obtained from the Duke University Institutional Review Board.

Measures. Facility-level data were extracted from public-access Medicare Nursing Home Profiles appropriate to the timeframe of the study.15 Data included the overall star rating (range 1–5) created by the CMS for consumers to use in comparing above and below average quality of care across NHs.6 These data were augmented by the 7 additional facility-related survey items submitted to DONs, which included tenure (in months) in DON position, percent of annual staff turnover, availability of a wound nurse (yes, no), policies on seeking and accepting wound patients, current number of PrIs (yes, no by patient), and current major projects such as new administration or major staff change.

Participating staff-level variables included age, gender, ethnicity, race, level of education, years/months in current position and any other position at current facility, years/months in a similar position at previous facility, employment status (full or part time), and usual working shift. 

NCAT administration. The NCAT was used to survey staff perceptions of nursing culture in each facility. The NCAT has been described extensively as a stable, valid, and reliable measure of nursing’s occupational subculture in LTC settings.1,2 Briefly, its 19 declarative statements are scored on a 4-point ordinal scale (total score range 19–76) and converted to normative ranking percentages (0% to 100%)1  associated with 6 factors that include expectations, behavior, satisfaction, teamwork, communication, and professional commitment related to nursing practice.2  Higher scores represent a positively toned nursing occupational subculture with a more beneficial influence expected when implementing a planned change in care.1 Normative rankings are based on data aggregated from 1025 staff members across 54 NHs and represent the percentage of those NHs whose mean score was less than the relevant level.1 Generally, cumulative rank percentiles are referenced in tertiles as high (> 66%), moderate (33%–66%), and low (<33%). For this study, NCAT items were prefaced as follows: “Please think about the practices in place to prevent pressure ulcers in your health care organization. Also, think about your work and professional commitment as you fill out this questionnaire.”

Focus groups. Focus groups were conducted in a private location in each facility during regular working hours by 2 study team members experienced in qualitative research. Facilitators used an interview guide to elicit participants’ perceptions. Each focus group was between 45 and 50 minutes and began with a consent process, followed by a description of the NCAT’s purpose and a presentation of the analysis of the facility’s results. The presentation included a line drawing of a prone human figure with 3 areas of PrI risk highlighted, followed by 7 pie charts, one each for the total NCAT and each subscale normative scores. 

Participants reviewed and discussed the results of the NCAT assessment for their facility and their implications for PrI prevention and care. Participants were asked their interpretation of the extent, if any, to which the total NCAT and subscales’ scores represented the culture of nursing related to their facility’s PrI prevention practices, including staff activities, attitudes, and experiences. Sessions were audiorecorded and professionally transcribed verbatim. 

Analytic strategy. Frequency and percent of NH profile items, DON items, and raw and standardized NCAT subscale and total scores were analyzed using descriptive univariate analyses. From these data, graphic presentations were prepared for the focus group discussions, including a line drawing of pressure points on a human figure and pie charts representing standardized total NCAT and subscale scores. Quantitative analyses also included analyses of variance with Bonferroni post hoc pairwise comparisons to ascertain whether NCAT total and subscale scores varied significantly by facility. 

Focus group transcripts were analyzed using NVivo 9 (QSR International, Melbourne, Australia).16 A systematic process for qualitative data analysis was employed to identify core concepts emerging from the focus groups using the NCAT theoretical framework to guide the qualitative content analysis. The analytic plan derived both a priori codes from the theory and allowed for new themes to emerge post hoc. 

Each transcript was considered a unit of analysis, and handwritten notes were made on general themes. The interdisciplinary focus group analysis team included 3 investigators with backgrounds in gerontological nursing, education, and organizational science. To establish consistency among the investigators during coding, all team members coded the same data excerpts from 1 transcript, discussed the coding decisions, and refined code definitions. Next, each coder independently completed coding the remainder of the relevant text of the transcript, and the group of coders met to reinspect all codes. A codebook with definitions was developed to increase trustworthiness. Next, a pair of coders was assigned to each transcript and coded it independently using the established definitions for codes and themes. The coder pairs met to examine the coding that each had applied to the text and compared the coding with each other to establish reliability. Intercoder reliability was greater than 85%. To increase trustworthiness, the full coding team met repeatedly until the full group had reviewed all coded statements for consistency and accuracy of codes. Differences in coding were discussed and resolved by the team. Finally, the PI and 1 other coder met to examine and ensure that all transcripts and coding assignments had been appropriately completed according to correct definitions across group transcripts.  

The analysis approach used is well-suited to cross-sectional data17,18 and is particularly useful in health care research because it allows for a priori concepts (eg, salience, sustainability)19 to be combined with inductive analyses. It also creates an explicit audit trail in the data reductions within and between individual facilities and between analytic stages.20,21 

Results

Across the 4 facilities, 109 staff members participated. Characteristics of the 4 facilities are presented in Table 1. The NHs ranged in size from 60 to 125 beds, with 81% to 92% occupancy. According to Nursing Home Compare data,15 RNs in the largest of the 4 facilities (D) spent the most minutes per resident per day caring for residents (57 RNs for 125 certified beds); CNAs in the smallest NH (C) spent the most minutes per day caring for residents (204 CNAs for 60 certified beds). Overall star ratings ranged from above average (4) to much below average (1) based on the CMS multifactorial rating system.6 Other publicly available quality indicators varied across the 4 facilities, as shown. 

Facility DONs reported tenure in the current job of between 3 and 17 months. Average annual turnover for staff positions ranged from 5% to 17% for RNs, 5% to 12% for LPNs, and 5% to 44% for CNAs. One (1) wound care nurse was available to each of the 4 facilities. The most common new change/project was a new administration. In the largest NH (D), nearly 1 in 5 residents had a PrI at the time of the study. Regardless of the facility, staff participants who consented to complete the NCAT survey and attend focus groups were overwhelmingly female (94%) and self-reported either black (57%) or white (38%) race. No statistically significant differences were noted among staff participants at the 4 facilities on any demographic or work history variable with the exception of years at the organization. Participants in NH C (mean time at the facility 6.45 [range 0.17–25.00] years) had worked significantly longer at the facility than staff participants in NH D (mean time  1.3 [range 0.08–5.00] years; P = .012; data not in table).

Mean raw scores on the total NCAT and each subscale are presented in Table 2 for the study sample as a whole and by facility; for each raw score, its cumulative rank percentile is shown. Overall for the 109 staff participants, the mean raw score for the total NCAT was 59.40 (range 19–76), which was standardized as 43.6%, meaning that 43.6% of a large sample of US NHs scored at or below that threshold. The highest total mean score on the NCAT across the 4 facilities was in NH C; 89.1% of the normed sample of US NHs scored 63.64 or lower. In other words, NH C scored in the top 10.9% of NHs. The lowest total mean NCAT score was in NH B; 25.5% of the norming sample scored 55.46 or lower. Raw and rank percentiles are shown for each subscale and each facility. Where raw scores were so close to the threshold between percentiles that rounding changed the standardized percentile result, that change is noted. 

To explore the first research question regarding how differences in nursing culture aligned with quality of care, the authors first focused on the high and low ends of the NCAT total and subscale scores in each facility, specifically their relationship to NHs nationwide. The 2 highest and 2 lowest ranked subscales are noted for each facility. Even the lowest ranked subscales in NH C (Behavior and Professional Commitment) were in the highest tertile of NHs nationwide at 92.7% each. On the other hand, 2 NHs (B and D) demonstrated their second highest subscale (Communication [B] and Behavior [D]) in the middle tertile only and their lowest subscale scores (Behavior [B] and Satisfaction [D]) in the lowest tertile nationwide. Scores on the total NCAT and on 3 subscales were significantly different in NHs B and C  (not shown in table). For example, NH B scored significantly lower than the other 3 NHs, not only on the total NCAT (P = .016) but also on the Behavior (P = .018), Satisfaction (P = .027), and Teamwork (P = .001) subscales.

The authors then explored whether the NCAT and its subscales varied in the direction of the quality of PrI prevention and care, based on evidence from both public archives and DON reports of clinical- and facility-related factors — that is, were more positive scores on dimensions of nursing culture aligned with higher quality care scores. Significant differences in nursing culture were noted between NHs B and C. Overall, nursing culture was significantly stronger in NH C (top 10% of US NHs) than in NH B (lowest 25% of US NHs), the latter of which was twice as large and had more than twice as many residents. Compared to NH B, NH C also scored significantly higher on teamwork (100% standardized score), satisfaction (top 2%), and behavior (top 7%) in the facility where CNAs spent nearly twice the amount of time with residents, annual turnover was approximately one third as great, and the overall star rating was twice as high as in NH B. 

The second research question focused on the relationships between staff and nursing culture scores in the context of PrI prevention. Staff comments about their respective facility’s results tended to fall across 4 quadrants that specified agreement or disagreement with the facility’s high or low score in the context of PrI prevention. For example, regarding the overall NCAT score, a participant in NH A said, “I think the trust here is very strong and the tool reflects that” (agreement with high score). In NH B, 2 participants reflected disparate views: “…25%? Yeah, I get where they picked it up from, yeah” (agreement with low score) and “We’re doing better than what this is saying…because we do try to make sure that we can prevent pressure ulcers and do what we are supposed to do” (disagreement with low score). 

Focus group members’ comments often reflected a priori, theory-based themes related to PrI prevention and the NCAT subscale scores. Of the 6 dimensions of nursing culture, teamwork received the most comments (28), ranging from positive (“I’ve been impressed with the way the nurse and the CNAs work together, and they’re willing to let me know if they need help”) to negative (“I think we could be better.)” Some comments were related to changes in teamwork over time (“I think that teamwork has gotten better”); others explained a low score (“There are just a lot of changes going on with the facility, things that we’re having to put into place that we’re not used to. So, it’s just a gathering and getting things out and putting them together and working better together.”)

Communication also received significant attention (18 comments). The comments ranged from “Nursing staff have respectful discussions and clear information exchange regarding the pressure ulcers and the needs of residents” to “There needs to be improvement in communication between CNA and nurses when they notice a change.” Breakdowns in communication were attributed to both a lack of confidence among CNAs (“If the CNA is not confident, they’re afraid. They stand back, don’t present themselves, and won’t stand up to this person”) and also to the multiplicity of key informants (“A family member may tell you that a resident’s friend passed away, and then they won’t eat, and they stay in bed, and the CNA may not pass it to the nurse who may not put the whole picture together.”)

The remaining 4 dimensions of nursing culture each received between 9 and 11 comments. Role confusion with respect to expectations and behaviors was mentioned repeatedly. Several comments highlighted the challenge posed by residents resistant to position changes or to taking a medicine intended to promote wound healing, with the resulting frustration that CNAs are “held responsible for their wounds.” Others cited the challenge of clarifying roles among the staff, including the need for “constant education to make sure that [CNAs] report things when it’s just redness instead of after it’s opened up, and that the nurse follows up on that” or “We’re doing our portion of what we can do to prevent them – like turning them and keeping them dry and putting the cream on them – but it becomes the wound care nurse’s job to make sure one doesn’t get any bigger.” Participants uniformly agreed that PrI prevention was a professional commitment (“Part of our job!” “Everybody as a whole is committed.” “If I have one PrI then I’m not doing my job”).

Finally, participants expressed both positive and negative thoughts about the NCAT as contextualized for PrI prevention. Supportive comments included, “It’s pretty much capturing [the situation]”; “It’s picking that culture up”; and “There is nothing difficult about what you’re trying to get at.” However, some dissatisfaction with the tool also was expressed. Respondents noted that the individual NCAT items did not reference PrIs nor were items sufficiently specific regarding the dimensions of PrI prevention, such as turning, drying, and so on. Staff members conjectured whether the lack of direction in each NCAT item could have distorted the results of the study. One staff member noted that the same items could relate as easily to other quality measures, such as falls prevention. Although participants acknowledged that the general instructions at the beginning of the NCAT instructed respondents to focus on PrI when answering all of the questions, a few participants suggested that each individual NCAT item should include a reminder regarding PrI (or other relevant focus area) or that the general directions be rephrased with more specificity. 

Discussion

This study is the first to explore the viability of contextualizing assessment of LTC nursing culture for a specific care component, in this case for PrI prevention. PrIs occur most commonly in older adults21 and are responsible for chronic wounds, amputations, septic infections, and premature deaths.3 Direct treatment costs of a PrI are greater than prevention costs,22,23 making sustainable prevention-focused protocols in LTC settings critical. Given that PrI development is a quality indicator for poor or nonexistent nursing care, the context in which care is delivered should be of concern. Both organizational and occupational subcultures are steeped in psychological, sociological, and anthropological traditions and have long been of interest to organizational scientists and practitioners. The composite of these two constructs is a reflection of the totality of employees’ work experiences, and care norms embraced by all of the nursing staff can pose unique challenges to implementing change for quality care improvement. The occupational subculture of nursing exerts a latent influence on the nursing staff’s ability to deliver high quality PrI prevention care.2 Similarly, evidence from a large cross-sectional survey24 of LPNs showed the subculture also may contribute to the variation in staff behavior that leads to inconsistencies in practice related to PrI prevention best practice recommendations. 

Although not significantly different from other NHs with respect to nursing culture, NH D provided an interesting counter-example to NHs B and C. Specifically, it was also a large facility and rated 2 stars (as was NH C), but NH D reported lower RN and LPN turnover rates compared to either NHs B or C but shorter time in current positions than NH C. However, its PrI rate among high-risk, long-stay residents was 3 times higher than either NH B or C. Where clinical- and facility-related quality ratings were as mixed as NH D, it may not be surprising that standardized nursing culture scores also would be mixed: an overall nursing culture score in the bottom 38% of NHs nationwide and standardized subscale scores in the middle range for behavior, teamwork, communication, and professional commitment, high range for expectations, and low range for satisfaction. This example is interesting, given that low staff turnover is commonly believed to protect against poor teamwork. More importantly, this finding may be useful to change agents in NH D in that strong staff expectations could potentially be leveraged to address the weak link of staff satisfaction when planning a change in care protocols.

Three (3) of the LTC facilities in this study reported new administrations since the previous year, and the facility with longevity in the administration had the lowest PrI incidence rate. Licensed nurses and DONs (who are more conversant with the overall organizational culture) have reported via indepth, semistructured interviews that they provide the majority of leadership and influence over the day-to-day implementation of care delivered,25 and CNAs represent the largest portion of LTC nursing staff and do the majority of the hands-on-care, according to large qualitative and quantitative studies.26,27 The relationships among administrative stability, nursing culture, and PrI require more targeted examination than was possible in this study. 

Although this study suggests that, when planning for a specific quality improvement project such as a PrI prevention program, administration of the NCAT provides information about which domain(s) of nursing culture are strongest and weakest in a facility, it remains to future studies to determine whether and how such information can serve effectively to guide tactical targets for designing the desired project. Low teamwork scores may require very different strategies for remediation than low professional commitment scores. Likewise, strong expectations or professional commitment may be leveraged differently to improve specific quality-of-care goals. Future tests may hypothesize that this diagnostic value to the baseline NCAT assists planners to avoid more scattershot and less effective approaches to change strategies in both the short and long term. Rigorous prospective cohort studies28,29 suggest that sustaining improvements to care over time has been challenging, with self-reported continuation often less than 50%.30 The current authors suggest that understanding the nursing culture before implementing change strategies may provide a window into the “black box” of sustainability, which has been largely neglected by relevant research.31 In the US LTC market where resources to provide dedicated program staff are scarce and leadership turnover rates can exceed 100% per year,32 additional data to guide sustainability planning are urgently needed. 

Finally, feedback from focus group members suggested that measures of nursing culture should be contextualized as clearly and repeatedly as possible, depending on the target quality care measure to be improved. Initial directions for completing the NCAT should be as specific as possible with respect to care behaviors. Specific items also should refer to the context of care in order to support contextually valid and reliable responses.   

Limitations

Limitations of the current study include the lack of reproducibility and causal inference inherent in a small, mixed-method study. Although additional research projects that link the NCAT to staff feedback would be useful, the authors believe broader-based LTC industry policy shifts that mandate both standardized measures of nursing culture and also confidential channels for staff feedback on the results of those measures would provide the most representative base of evidence for generating testable hypotheses of whether nursing culture affects PrI prevention directly.

Conclusion

The purpose of the study was to evaluate the relevance of a LTC facility’s nursing culture to both its quality measures and its staff’s perceptions of care in the context of PrI prevention. The findings from the NCAT and focus groups suggest weaker nursing cultures at the facility level may be associated with greater prevalence of PrIs and with the staff’s perceptions of barriers to and facilitators of the implementation of new care practices such as repositioning residents to prevent PrIs. The implications of this study are that research on effective workplace change in LTC settings requires a more nuanced understanding the meaning and impact of the nursing culture on worker performance33 and that further examination of these relationships may yield new perspective on the values and beliefs that underlie the perceptions of frontline workers in LTC facilities. ν

 

Acknowledgments

Scientific editing was provided by Judith C. Hays, RN, PhD.