Rabbi Ilan Haber

From Conversations: Readers Respond to A Review of Kiruv

Why We Should Stop Measuring Success: A Contrarian’s Perspective

It is no surprise that one of the key questions regarding the state of kiruv, and thus addressed in the past issue of Klal Perspectives (Fall 2012), revolves around its measures of success.  The notion of measuring success is quite in vogue, not only within the kiruv movement, but also within the entire sphere of non-profit endeavor. The term is taken from the annals of corporate experience, from which many of the key donors and supporters of kiruv hail. In a for-profit environment, corporations are expected to have key measures of their own growth or success as a means of focusing their efforts creating better efficiencies and achieving profitability. This is true whether the measures are net profit, in the classic sense, or units sold, profit margin, stores per square mile, etc. However, the use of measures of success within a non-profit environment, though emerging from the best of intentions as evidenced in the kiruv enterprise, is often distracting, irrelevant, and at times, counter-productive.

Although the care with which a kiruv organization measures success is often taken as evidence of its strategic clarity, in my experience it is anything but. Recently, I interviewed an individual who was applying for an outreach position, and was asked, “What were my three measures of success?” Though I do not quite know where the number three came from, my response was that I arrive at the question from a completely different standpoint. The issue, as I see it in my own organization, and as I recommend that other organizations view it, is not “what are my measures of success?” but “how do we better accomplish our mission?”

For me, evaluation must be a learning process that helps me plan and prepare better for the future. It should enable me to understand, with complexity, what it is that I am actually doing, what I am willing to be accountable for, how we ensure that we continue to do what we do well, and how we improve upon our work.  It is the starting point of a conversation, not the end of one.  It is part of a process that is rooted in a fundamental, unsentimental, and brutal commitment towards excellence. Its philosophical underpinnings is “cheshbon hanefesh” (self-evaluation), but as applied to an organizational context. In cheshbon hanefesh, we examine who we are, what are we engaged in, if our endeavors are worthwhile or fruitful, what challenges are endangering our progress, etc.  We may keep track of our mitzvot and aveirot, not as a quantitative accounting and scorecard to see for ourselves or to prove to others whether or not we are a tzaddik, but as symptoms of more pervasive and fundamental progress.

Measuring success gives us an answer, but I would much rather have a series of questions. To understand better, I suggest we look at the following example. In team sports, let’s say football, there is inevitably a winner and loser, as well as a quantitative measure of success, otherwise known as the score. However, knowing that one team defeated the other by a score of 38 to 35, while giving us a quantitative measure of success, reveals nothing about how the team should prepare for the next game. It does not tell us why one team scored 38 points and the other 35. Did the special teams play well? How were the teams coached? Did home field advantage or the weather play a role? What is the morale in the locker room? Was the game plan prepared and then executed effectively? These are the questions that, when asked, measured, and in turn addressed, enable a team to more effectively meet future challenges.  In the end, the final score is actually irrelevant except to fans and statisticians. When focused on broadly and effectively measuring and noting “process” as opposed to “product,” one is able to put one’s evaluation into action in order to perform better.

Of course, this assumes that one is actually capable of arriving at a clear, quantitative measure that is directly caused by or at least correlated with or one’s efforts. Outside of laboratory conditions – in other words, in any social and open environment – it is almost impossible reliably to assess how a particular outcome was affected by a particular set of actions. There are simply too many variables to be accounted for.

For example, let’s say that a specific person did in fact become Shomer Shabbat, and that person was indeed proactively approached and engaged by a mekarev (kiruv professional). Who is to say that the actions of the mekarev made the difference? What was the role of the baal teshuva’s upbringing? What is his or her mental state? Were there other people that interacted with the baal teshuva, influencing and encouraging his or her actions? What was the role of community? Did the spouse of the mekarev play a role? And on and on… If this is true for each individual person counted on a spreadsheet as a measure of success, the reliability of such measures becomes extremely suspect when multiplied by many such individuals. As an aggregate, such numbers, if not rooted in careful process evaluation that addresses or at least accounts for each of the mitigating factors, are extremely suspect, tend to be misleading and are ultimately unusable as a planning tool for the future. Investors know this phenomenon as “past results do not guarantee future performance.” Pushing mekarvim to focus on and produce results, which are then utilized to measure their success, can have at the very least unintended negative consequences that accompany the positive, and at worst, can compromise the entire endeavor of the mekarev.

In the past issue of Klal Perspectives, Rabbi Shaya Karlinsky astutely pointed out that single-mindedly focusing on the ends in kiruv, as opposed to, or sometimes at the expense of, the means, can often have tragic or catastrophic consequences. He articulates that the entire crisis enterprise and emphasis on quantitative results encourages the mekarev to focus on bringing in new “customers” at the expense of “customer service.”  I do not know of any kiruv organization that measures how many students are turned off, or taught to have disdain for Torah and frumkeit, due to overly aggressive tactics.

Moreover, instilling fear and urgency in the activities of the mekarev through correlating their job security with their quantitative success, as if their own commitment to mesirut nefesh (self sacrifice) is not sufficient, encourages them to be dishonest in their reporting.  If one is worried that they are not working hard enough, then hire better. Or at the very least, utilize supervision, personnel evaluation, and professional development as tools for motivation and performance that are more effective than simply measuring their success.

The focus on quantitative measures of success is actually symptomatic of a much deeper malaise in Jewish communal service. We are so starved for any information that helps us understand our environment and context that we are prone to seize upon any data and utilize it ineffectively. Any study that comes out is immediately digested, without any critical review or context, and utilized to affirm what we do, or to make changes in our approach. Social Science research, no matter how worthwhile, is going to have a specific context, its own share of methodological flaws (there are no foolproof evaluation techniques), and is typically going to provide us with but a momentary glimpse – a snapshot in time – of complex and dynamic sociological phenomena.