You can't square a circle by improving your tools. Conceptual muddles can't be resolved with more data. To seek one standard by which all nonprofits can be measured is to seek what does not exist. A wealthy Rapture Ready Christian, and a prominent donor here in Dallas, TX, once told me, in front of witnesses, who took notes and nodded sagely, hoping to flatter him out of some ready money, "Diogenes, we should not give for causes, like environmental protection, that work against the Will of God." That is, God has slated planet earth for destruction. Christians will be raptured to Heaven. Liberals will be left behind to die deaths as horrible as those inflicted by General Sherman on the flower of southern chivalry in the War of Northern Aggression. Each environmental disaster should be embraced as bringing the Second Coming of Christ closer. Now, show me the metric - the one metric - on which to place this man's favorite charity (say, a Ministry devoted to converting homosexuals through the use of electroshock) and the Sierra Club, or Astraea. There is no one metric because there is no one vision of the good. There are many visions of the good and at any given moment they are at war with each other, each characterizing the other's god as the devil.
Objective evaluations of charity are the leading edge of 'Pataphysics, Alfred Jarry's "science of imaginary solutions. " We might as well weigh sunbeams in ounces and divide by cucumbers in inches to arrive at a dollar amount.
The conversation about metrics is well summarized by Sean at Tactical Philanthropy.
There are many good questions that metrics can answer -- e.g., Did the rate of new HIV infections drop after intervention X? It remains the case that some of the most important questions for donors will lie forever beyond the reach of measurement. We need the wisdom to discern between these two kinds of cases. An ethical, professional evaluator will have and share this wisdom.
The desire to measure springs from an enormous frustration with what some commentators have called "philanthropy as usual." I share some of these frustrations, but I also believe traditional philanthropy has been grotesquely caricatured. It's in many ways an easy mark.
Posted by: Albert Ruesga | March 31, 2007 at 11:29 AM
Thanks, Albert, you are operating here with an unfair advantage - you know what you are talking about. I am not in the grant-making world, so my views are superficial, and I have no real stake in it, from day to day, so I can be supercilious. Clearly, any organization should have ends in view and benchmarks to indicate progress towards goals, or results achieved. And once those benchmarks are in place, you can raise the questions about effectiveness and efficiency. That would be one way to do an evaluation of any organization - does it effectively and efficiently achieve the ends in view that it sets for itself - whatever those ends might be. Thus, a "good" or "excellent" charity might be one that we personally detest, since it effectively and efficiently advances an agenda with which we disagree.
Posted by: Phil | March 31, 2007 at 11:35 AM
No one in any part of this debate has proposed to measure all good on a single scale. You're arguing with a straw man.
I urge you to engage in the issues that are actually under discussion, and acknowledge - and/or comment on - the actual approaches to metrics that are on the table.
Posted by: Holden | March 31, 2007 at 11:38 AM
I agree with you about arguing mostly against a straw man. The concept of metrics is just not one that engages me at an emotional level. I have worked in jobs where endless efforts were made "to measure results," when our input into a situation was minor and that input was swamped by other factors. At best a charitable effort is a contributing factor, along with myriad other direct and indirect effects. To force a charity to manage and measure makes good business sense, of course, and must be done, lest the situation be totally fluid and unmanaged, but to expect the measurements to be truly illuminating is to take a pretty myopic view. Say, we measure the number of people who borrow from a microfinance lender, and who go on to start tiny businesses. And say, when you go into those towns the end result is an ever rising disparity between have and have not. So, you are helping some while others fall farther behind. Or, you discover that the those helped become supporters of what is in fact an oppressive regime, since they have a stake in the status quo. What then is the measure? What is lacking here is wisdom, judgement, humility. The business mind measuring and managing just may not have sufficient subtlety even to appreciate the issues (spiritual, ethical, pyschological, sociological, economic, governmental, political) that are in play. The arrogance is what is annoying. You can't measure the effectiveness of your own life on earth, so how can you measure the effectiveness of an organization? How can you prove that your days on earth have been a net positive to the universe? Prove it!
Posted by: Phil | March 31, 2007 at 11:49 AM
I can't prove it, or anything. The difference between academia and business is the difference between proving and betting.
We have no choice but to bet - or simply stop donating altogether. As long as we're betting, we do it as well as we can. "As well as we can" might still be a 2 out of 10, but that doesn't make the effort any less valuable or essential.
You're right that none of these metrics will ever really tell us what we want to know. But (as you acknowledge) we still need to use them. That's why they're worth discussing.
Posted by: Holden | March 31, 2007 at 12:49 PM
I agree. Betting is a good way to say, placing a better bet. A lot of that has to do, don't you think, as in business, with the confidence you have in the people and their judgment? When you know them and trust them, and appreciate that they are experienced and engaged and driven to do what is right, that is a good bet too, even absent clear short term "results" and metrics. Metrics can be impersonal, and shift power to the donor. At some point, maybe really get close to an org, and getting personally engaged as fellow servant in the work would produce the best results and the greatest satisfaction. Dollars in and results out, is one way to look at it, but when you get engaged as a volunteer, one result may be self-transformation too, a more satisfying day to day life, making some gains and working with those who share an ideal.
Posted by: Phil | March 31, 2007 at 02:07 PM
Both (using impersonal metrics and personal contacts) are good ways to improve your betting, and there's no need to choose one or the other.
Businesses get financed through personal contacts, people funding people they believe in; they also get financed through the public capital markets, where investors have shallower (more metrics-driven) knowledge but more options. Both are extremely valuable in allocating capital. The nonprofit sector currently has the former and lacks the latter.
Posted by: Holden | April 01, 2007 at 05:03 PM
Yes, right. You can measure a company's bottom line with audited financial statements. You can't really measure so well a firm's "social benefit." People who grow up with business or accounting as their reference point always want metrics, and generally get metaphors, signs, and symbols. You don't accept that some ratings, like those for movies, restaurants, and literature, remain matters of good judgment, and a trained eye?
Posted by: Phil | April 01, 2007 at 06:24 PM
We are agreed that many tools may be applied, but it would not be good to stop there. When is it appropriate to apply a metric, and when is another method required? I claim that metrics only make sense after you have a model, and you can use those metrics to explore and calibrate the model, and having a model implies you are well down the road of understanding the underlying problem.
On the flip side, if you don't have metrics, it is hard to form good models, and implied here is that sometimes metrics are a research tool and other times for feedback and tuning of a well understood process.
It would be interesting to interlace the conversation about aligning foundation investments with their charitable purposes. It's an open question, right? How do you create a market in good works? Should we?
On the other hand it would be very useful to have venture type investments designed to bring new ideas to a point of scaling for capacity, a point at which more traditional foundations could invest in expanding the capacity of said proven idea and creating a limited payoff for the venture risk takers.
In some senses there already are markets in basic services (commodified social services) that are services by non-profits and funded by foundations. Introducing performance metrics in that space might do a lot to improve things, but you can never lose sight of the big picture. If one investment's impact is creating social harm and being serviced by a complimentary social investment, there is a potential for vicious spirals out of control. In other words you must be careful about what is measured and how it is rewarded or not.
Posted by: Gerry | April 01, 2007 at 06:52 PM
I like what you say, Gerry, about needing a model, or theory of social change, for metrics to be meaningful, and likewise how hard it is to have a model, when metrics are missing.
Posted by: Phil | April 01, 2007 at 09:34 PM
Yes, except what still troubles me, is that we have models built up from metrics and measurement and them models shaping outcomes which in turn drives changes in the system and the outcomes expressed in metrics. This is a feedback cycle, and jut because one set of models works doesn't mean there aren't another set of models with related systems configurations (higher level emergent structures) that is just as coherent as the first set. These may take the same or similar terms and arrange them in different ways, but they could have very divergent dynamics and outcomes and it would be a mistake to take some principle derived and applied within in set for the other.
Posted by: Gerry | April 02, 2007 at 09:50 AM
Like confirmation of a theory in science? Based on limited and theory-bound data? Measuring phlogiston?
For years urban planning was based on measurable reductions in "the miasma" that was said to cause cholera. Data to the effect that it was all about sewage contaminated water was disregarded for years. The miasma theory fit the data, and who needed to upset everything with a new theory that might be inconvenient in a myriad of ways to the city planner, property owners, etc. Miasma was caused by the poor huddling close. Sewage came from upstream.
Looking upstream can be dangerous.
Posted by: Phil | April 02, 2007 at 08:09 PM
THE FOR-PROFIT SECTOR IS NOT EASY TO MEASURE.
Why do people think it is? So you can see what a firm earned. So what? That doesn't help you decide whether to buy their stock. Of course you know the goal - profits - but how in the world do you predict profits 10 years out for a growth company? Is there a formula for that? (There is not.) And of course, if you as an investor can't come up with things that others can't see, you're worthless - that's a problem that donors don't have to deal with at all, and it ensures that anything easily measured is ipso facto useless to a for-profit investor.
For-profit people are obsessed with measurement not because it's any easier for them, but because they have swifter and stronger punishment for failing. They suffer immediately when they fail and they benefit immediately when they succeed, and they learn quickly about the inescapable value of evaluation, no matter how difficult and incomplete and frustrating. Evaluation IS NOT EASIER in the for-profit sector, and most people do not enter the sector excited about spending their time and resources on it, but unlike at charities where quality of programs and success of fundraising are almost entirely separate, these people get a kick in the pants early on.
To a lesser extent, the same goes for science, also not as secure and easy and measurable as you make it out to be. Yes, measurement is hard. Yes, you can never really prove anything. Yes, you can go along one intellectual path for 50 years, or 1000 years, only to discover that you were missing something right in front of your nose the whole time. All of these things have happened. And all of these points are for academics, not gamblers. They pollute the theoretical cleanliness of the case for evaluation - not the indisputable practical value. They are relevant to the unimportant question, "Will metrics assure us that we are doing things right?", to which the answer is no. They are irrelevant to the important question, "Should we do formal evaluation or not?", to which the answer is a resounding yes.
Posted by: Holden | April 02, 2007 at 11:59 PM
Nobody said measurement was easy. Every company I've worked for measured all sorts of things, and spent all sorts of money on it, and only a small part of that was every worth anything. How do you measure whether the measurement is measuring anything? I'm sure that many of those managers would agree with your "resounding yes", after all their jobs are dependent on measuring and evaluating whether the process the organization is following effective or not.
I'm not saying there isn't a place for it, but it is far from obvious that it is necessary in all cases. I'm predicting that it is most effective when the problem space is well traveled.
The bottom line is that there is too much focus on the bottom line in the for-profit as well as the non-profit world and not enough focus on the big picture. Yes an evaluation that takes this into account would be useful, but it will also be more like poetry than statistics.
BTW, there is a good predictor of company profitability five or ten years out, the quality of the people in the organization. Is there an effective procedure for evaluating a management team for quality, or even specifying what qualities that evaluation would look for? It is well and good to say this, but it is another thing to do it competently. That is Warren Buffet's investment strategy, find undervalued companies with good managers, which works if you have Warren Buffet, and as you have pointed out, there are clear indicators of success and failure here.
Whatever its limitation as a measure of value, unidimensional valuation by profit and loss at least makes the winners and losers obvious.
Posted by: Gerry | April 03, 2007 at 01:46 AM
There is an old Latin tag, "Do not attempt to fix what you do not understand." Unless you understand the issue area, whether education in a city, or health, or social justice, or whatever it might be, how can you know what to measure? What is symptom, what is cause? What are the leverage points? What are the indirect effects? And how much time will a donor take in mastering the issue area? As Albert notes at White Courtesy Telephone, donors want benchmarks so benchmarks are offered until one clicks with the donor and the money flows.
Posted by: Phil | April 03, 2007 at 08:47 AM
Phil - What do you mean by not fixing what you don't understand? I see two possibilities.
If you mean the charities/organizations themselves, I agree with you. Donors should not try to fix the charities they donate to. They should try to understand them. That is absolutely impossible unless the charities are doing self-evaluation. On this, I speak from experience.
If you mean the PROBLEMS that charities address, then it sounds like you are asking people not to donate. Which I would be surprised, but not knocked off my chair, to read with the we-can't-do-it-perfectly-so-let's-throw-up-our-hands attitude I've been seeing.
Gerry - NOBODY likes evaluating themselves. And everybody likes evaluating others. Nothing along these lines can explain the reason that for-profits do so much more evaluation than nonprofits.
The difference is that if these evaluations are really a waste of money, the companies that skip them will destroy the companies that do them; if they are necessary, the reverse will happen. Thus, in the for-profit sector (but NOT in the nonprofit sector), you can use what people commonly do as actual evidence of what commonly leads to success. Of course some evaluation is excessive. But the fact that for-profits so consistently and so extremely do more evaluation than nonprofits suggests simply that the right level is far above what nonprofits do.
Posted by: Holden | April 03, 2007 at 09:29 AM
As to using the quality of people as evidence, I agree that it is one of the best and most useful pieces of evidence you can have. 100% worth doing. But you are really suggesting that it's sufficient by itself?
People are good at all kinds of different things. Chatting with someone gives me extremely limited information about how good they are ... seeing what they've accomplished (and how) is a far better indicator.
There are lots of smart and well-intentioned people out there. Without evidence of accomplishments, how do you decide between them? By holding a conversation contest? This is analogous to the question of how you decide how much Google's stock is worth, once you've established that its founders are smart guys. The answer: you need more info, period.
Posted by: Holden | April 03, 2007 at 09:42 AM
"Don't fix what you don't understand" counsels humility in the face of social issues that are the resultant of many forces. You could spend a lifetime studying a single issue, like homelessness, from the perspective of public policy, charity, mental health, poverty, bankruptcy, causes of bankruptcy, legislation about credit, and on and on. Where in those systems to best target efforts and funds? To stand outside the efforts of those who have spent a good part of their lives working in that area and to call for objective measurements seems overbearing or managerial. Why not listen, learn, get engaged, and see what emerges with growing personal experience and first hand knowledge, before making the speech, an easy speech to make, about metrics, results, etc? "First do no harm," would be another ancient saying to the same effect. I am not in the nonprofit sector, am neither a grant maker, nor a grant receiver, so my own expertise her is limited. I would take seriously - much more seriously than my own ideas - those of Albert Ruesga at White Courtesy Telephone. He is speaking from experience.
Posted by: Phil | April 03, 2007 at 06:11 PM
If there were exactly one reputable charitable organization, you would at least have an argument that could be considered, though I'd still disagree. As it stands, there are literally over a million charities to choose from and if I narrow them down to only the largest, most reputable ones directly attacking the problems I care about ... that leaves a few thousand. And they're all using different strategies and asking for the same money.
So I can donate without understanding, i.e., fling my money about at random; or I can refuse to donate a thing until I truly understand the issues (never); or I can do what a good gambler does and always act on my best guess, while always seeking more information.
Posted by: Holden | April 04, 2007 at 12:52 AM
Holden, I would suggest that you fall in love with a charity, get passionate about their mission, volunteer, and get your brains and experience working for their cause, and let the money follow.
Posted by: Phil | April 04, 2007 at 08:33 AM
So from the thousands that could fit, pick one at random? Is that your suggestion?
Generally falling in love with someone or something comes AFTER getting to know them ...
As long as people continue to think along the lines above, there will continue to be no mechanism for better charities to outlive/outlast worse charities. And so, the mere fact that a charity is reputable will continue to be no indication of its relative quality.
"But they're all doing GOOD work," you might say. Not true, but even if it were - the difference between good and best is just too enormous to leave it at that.
Nothing else in the world works the way you're describing, and anything that did (with the mere fact of "experience" taken as indication of qualification) would be a disaster.
We owe the needy a better effort than this.
Posted by: Holden | April 04, 2007 at 09:51 AM
When you go searching for a life partner, do you first ask them to do a self-evaluation? Maybe you do, there are many approaches to these situations.
Posted by: Gerry | April 04, 2007 at 12:38 PM
When I go searching for a charity, I'm looking for an organization that gets results. So I want to see some of those results before committing. To answer your question:
When I go searching for a life partner, I'm looking (among other things) for someone I like spending time with. So I first generally take the approach of going on a date, then going on another one contingent on how the first one went, then forming a more intimate relationship and seeing how that goes before making a full commitment. I had thought this was a common model.
Posted by: Holden | April 04, 2007 at 04:23 PM
"But measuring how much you like someone is inherently impossible!"
"How can you go on dates with three women and decide which you like best? Doesn't that involve converting your feelings for each to a single scale?"
"Yes, measurement is all well and good. But in the end it's unreliable. You can date someone for four years, without ever finding out about how she'll deal with children - and then you'll regret ever having wasted your energy on the evaluation!"
"You can never really know for sure whether someone is right for you. It takes a lifetime. I advise you to stop trying."
Hmmmm. Perhaps I should forget this idea of "testing" a woman before I marry her. Perhaps it would be best to just pick one and commit, say by clicking the prettiest one I see on PlentyOfFish. Or, even better, get a personal reference - find someone I trust, ask them whom I should marry, and let that be the end of the decision.
Sorry to be polemical, but I'm trying to illustrate how your suggestions sound to me. Just as my experience with a woman is essential evidence in determining whether I like her, evidence of how much a charity helps people is essential evidence in determining how much it helps people.
Posted by: Holden | April 04, 2007 at 04:46 PM
We do some things for results, after a rational decision process. But some of the most important commitments we make are from the heart, and fortuitous. Marriage grows most often from chance encounters. And marriages stay together because we ask, "What can I do for you now?" Rather than, "What have you done for me lately?" Or, "What have you accomplished lately?" Results are sometimes what we achieve together with partners. That might be a model to pursue in your philanthropic work. Think of it as finding a partner in the work you care passionately about, rather than doing through a rational decision matrix. Not "the best possible partner," but a good one, one to which you can make a tentative commitment, then a deeper and deeper one. In other words, consider dropping the stance of objective third party and consider becoming a passionate partner.
Posted by: Phil | April 04, 2007 at 07:01 PM
Again, it comes down to the fact that a relationship and a charity have different purposes. I don't need evidence that a woman has helped others, only that she makes me feel a certain way. That evidence need not be anything other than experiential/intuitive. By contrast, I don't need a charity to make me feel a certain way, but I do want it to help others, so I need some sort of evidence that that has actually happened.
Experiential/intuitive evidence doesn't cut it because what I desire in a charity goes beyond how it makes me feel.
Another breakdown in this analogy is monogamy. As a donor, I offer money; there is good reason for loyalty and "stickiness" in volunteering, and in relationships, but there is no advantage to it in donating.
So the analogy isn't perfect because they're different goals, and therefore different kinds of relevant evidence, and that means different kinds of investigations. What is exactly right about the analogy is that no one in their right mind picks a life partner without accumulating a heck of a lot of evidence, evidence that is both backward-looking and relevant in its way.
Posted by: Holden | April 04, 2007 at 07:33 PM
The angel knocks on the barred door once, Holden, and passes on.
Posted by: Phil | April 04, 2007 at 09:11 PM
What sort of donor thinks like this? What is the motivation for giving if not to express the highest and best values of the giver? If you succeed in turning philanthropy into a market for good works, how is that really any different than turning marriage into a market for domestic and reproductive services?
The people that I respect, who are doing good works have no time for this sort of thing. The work is its own reward, but to be effective in this monetized world, sometimes you need cash. A donor who wants to be effective can easily find the important champions of any cause they say they care about. If these champions are well funded they attract new talent to the cause, and give them the opportunity to become champions as well. Figuring out how to fund them is the donor's only concern, and I would think it would be best done in close consultation with those champions.
How about another analogy. What you suggest is like handing a potential CEO an employment application. You want to be effective, you find leaders and collaborate with them to create a financial plan for a particular activity. Decide together how you will know whether it is worth investing further resources in that activity. If you want to collect good data, you will have to build data collection into the operational processes of the organization.
The small, often community based organizations that are already doing the good works are all staved for funds, while managerial class mavens debate how to divvy up the money. All this evaluation you speak of is just more overhead at the point of service to the community. Collecting good data is valuable to everyone, and that data is most valuable if it is widely shared (for the cost of reproduction ;-)
Posted by: Gerry | April 05, 2007 at 04:44 AM
The core statement I disagree with is this:
"A donor who wants to be effective can easily find the important champions of any cause they say they care about."
So maybe you know something I don't. What I would like most like to do - and I'll be narrower than usual about what I value - is fight the achievement gap in the U.S. I want children in disadvantaged/poor communities to end up as fully empowered adults, contributing to society, with all the advantages I've had. Kindly point me toward the important champions and explain how I can easily confirm that they are in fact the right ones.
As a side note, I think this question answers itself: "If you succeed in turning philanthropy into a market for good works, how is that really any different than turning marriage into a market for domestic and reproductive services?" Domestic and reproductive services aren't the defining goal of marriage. Good works are the defining goal of philanthropy.
(So it's OK if the sphere of philanthropy sees relationships suffer, but good works increase.)
Posted by: Holden | April 05, 2007 at 10:37 AM
Are you saying that this is a core interest of yours and you don't know how to do this without using more or less formal evaluations? Then I would wonder just how well you are applying yourself. Get out of the office and talk to some people who are doing this work, and I think you will have a better idea of who is who in your field of interest.
I'm speaking from experience in the areas that interest me, and in just a few years of looking and listening and I have more than enough information to plot an initiative or three. Only problem is a don't have a trust fund or a foundation's money to spend on it, and I have to watch clueless bean counters try to justify their managerial processes.
Just how much overhead is involved in these evaluations you advocate? How does that contribute to the bottom line of good works? How much should I spend on doing these evaluations before getting to diminishing returns?
Posted by: Gerry | April 05, 2007 at 06:36 PM
The overhead question can't be answered in the abstract, but we can get a ballpark sense from looking at what works in business. That would imply that evaluation is worth a LOT of overhead.
I don't understand your questions about core interest. This has become a core interest of mine because it was impossible for me to donate well without turning it into one. But initially, what I said was: "I'm not a nonprofit guy, I'm a finance guy. I don't have time, I have money. I want to use that money to help disadvantaged people get the opportunities I've had. How can I best put it to work?"
What I found was thousands of reputable organizations in this area, and no easy way to decide between them. When you say it's easy to identify the important champions of whatever cause I name ... that implies that you can answer my question. Please do.
Posted by: Holden | April 05, 2007 at 06:46 PM
I wasn't clear that it is your money you want to invest in good works. Some of my pointier points are really directed at the same thinking in the hands of a foundation grant-maker, I apologize if I pressed some to hard.
To some extent it depends on the scale of your giving. Phil can expand further, but he started this site thinking about how to coach givers, to start with the conversations about who you are and what you value, and eventually leading to forming plans for actions and gifts. The mechanics, legalities and financial will depend on scale, if you have a small or modest amount to give, you're probably better off going with a mutual fund approach, give to established charities and foundations that have low overhead (high percentage going to programs), and/or giving to local causes that move you personally. At some point the overhead of creating personal or family trusts is smaller than the gains that can be made. That's where the coach would help the donor form a team of professionals with the skills to do what you need.
This is my summary of what Phil has been working on for some time, creating a network that can do this. If you have a sizable gift fund, please contact Phil privately to get the detailed lowdown on what they can do and when.
I still think you will be happier and more effective if you get personally involved in the gifts that you make. I know time is precious, I have two kids and a more than full time job, and I can take the time to connect with the causes that move me. If I had money, I would likely be entrepreneurial with it. I would invest in some people I already know. Some of the ventures might even be profitable and they for sure would be positive for the community.
Posted by: Gerry | April 05, 2007 at 09:09 PM
Holden, have you thought about, say, mentoring? Dan Bassill at Tutor Mentor Connection might have some suggestions.
http://www.tutormentorconnection.org. He frequents this site from time to time. He might be able to point you to a program you could investigate yourself or get involved with as a donor or a mentor. Your goal speaks very well of you. I hope you connect on terms that are meaningful for you with a program you can support and see the results you want.
Posted by: Phil | April 05, 2007 at 09:53 PM
Holden, another idea would be to join a local Social Venture Parnternship. They are groups of business people who pool a little money, some time and some volunteer effort to partner with local nonprofits. Part of that effort is doing site visits, working with the nonprofit's staff and board, or otherwise putting time in along with money. These SVPs also have an ongoing donor (or as they call them "investor") education program. We have one in Dallas called Dallas Social Venture Partners (DSVP.org). We actually support k 1-12 education in Dallas. You would find yourself in an SVP surrounded by other "investors" who take a businesslike view of philanthropy. You are definitely not out there on the margins. Many feel as you do.
Posted by: Phil | April 05, 2007 at 09:56 PM
Here's exactly where I'm coming from:
For the last several years, I was in the situation I described: wanting to spend money, not time, to help the world. I didn't know then about SVPs (I do now), but I formed my own - after a fashion - last year by inviting my friends to join me in my investigations. The difference is that we wanted written, not experiential, evidence for a variety of reasons. One is that many of us were most interested in the neediest parts of the world, not local communities. (My personal top interest is the U.S., but I am also extremely interested in these areas, to the point where I could change my mind.)
Another, though, is that experiential evidence just can't answer the questions we REALLY want to answer. I want to help disadvantaged children have all the opportunities I've had. But is tutoring the best way to do that? Can that possibly compensate for a bad school system - what about reforming public schools along the lines of New Visions? Or saying to heck with them, and providing charter schools like KIPP? Or is the problem really less about the school system and more about children's communities and self-esteem? Should I fund after-school and recreational activities, like what Children's Aid Society does? But then, they say a big part of the problem is also basic health and nutrition ... how much of the problem?
I can't answer these questions by volunteering and getting to know people. The only way to answer them is by getting an idea of what ACTUALLY ENDS UP HAPPENING to children who get one kind of help or another. This is the information that needs to be documented ... in most cases charities simply take one of these many avenues to helping kids, and literally don't have any idea how it turns out; their only interaction with the kids is in the context of their programs.
We didn't want personal counseling on our personal values. Our values are too broad for that to be useful to us (I'm sure there are many others who feel differently). Figuring out whether I value tutoring or health care isn't a matter of searching myself - it's a matter of evaluating those activities, because whichever one helps children ultimately be ready to take on the world, that's the one I value.
So, as Gerry suggests, we wanted to take the mutual fund approach. Only problem is that there was no way whatsoever to evaluate the mutual funds along the axes that matter to us. Your suggestion to look for "low overhead" is shocking. Doing this neither narrows the field enough, nor has any meaning beyond knocking out the worst crooks. In any difficult and complex problem, ideas are more powerful than money, and that means paying for the infrastructure (including salaries) that helps a charity plan things well is important - pinching pennies from this in order to get more "straight to the programs" is throwing money at a problem. That's exactly what I don't want to do because I want to help solve the problems.
Having put in as much time as I've put in part-time, I've concluded that the information we need to collect has to be collected full-time, as a grantmaker. Grantmakers already collect it, of course, but they won't and don't share it with us. That's where the GiveWell project comes from: the idea is to be the world's first grantmaker that actually exists to be transparent, and serve individuals like my former self (and current friends) who just want more information on how they can translate their money into helping people. That's why I'm ready to leave a great job for a project that I think needs to happen.
That's my story. To be clear, I like my job. If you could convince me that what I'm doing is misguided or unnecessary, I'd happily return. If you could show me that identifying the "important champions" for what I want to do is easy, I'll just give to them, but you haven't done that so far. And if you can show me that volunteering and talking to people trumps evaluation - as frustrating and difficult and ultimately unreliable as it is - that will certainly affect how we go about doing our evaluations. So far you have done none of these things, but please don't respond "Holden, I think you are doing a wonderful thing" if you don't believe it because that isn't useful to me - tell me what you think.
Posted by: Holden | April 06, 2007 at 11:46 AM
OK, I do like and admire the direction you are taking, "That's where the GiveWell project comes from: the idea is to be the world's first grantmaker that actually exists to be transparent, and serve individuals like my former self (and current friends) who just want more information on how they can translate their money into helping people. That's why I'm ready to leave a great job for a project that I think needs to happen." I particularly like your emphasis on transparency.
Have you looked at Newdea.com? They may be the closest to what you are trying to do. They bring grant makers and grant seekers together around technology that allows the grant maker to see how the dollars given are being used, and the results (as reported by the recipient of the grant.) The founder of Newdea does have international grant making experience.
You can see from their tagline ("Proof of Impact") that they are appealing to donors who feel as you do. They are actually a forprofit.
If you do check them out, Holden, please let me know what you think. Newdea is one of the more original and ambitious efforts out there. I would be interested in your assessment.
Posted by: Phil | April 06, 2007 at 12:18 PM
Wow, just when I'd thought I'd seen them all ... haven't seen this, and it does look like it's along our values. I'm checking it out as we speak. Thanks for the link.
Posted by: Holden | April 06, 2007 at 01:51 PM
Tnanks, Holden, for persisting. I am interested in your observations as you did deeper.
Posted by: Phil | April 06, 2007 at 02:59 PM
OK, I made an account and took a look around my fields of interest. Here are my thoughts.
First off, this obviously can't address my needs in its current state. Searching for Education in the USA yielded 2 organizations, each with no documents attached (just a couple sentences on what they do). You have the common social network problem ... I won't use it till it's useful, but it won't be useful till people use it.
I also think the structure is a problem though. We originally conceived GiveWell along a similar model: build a tool that will allow the construction of a knowledge base by individual donors, almost like Wikipedia. Doing the work on our own, we found this impossible for two reasons:
1. The issues are very complex, there are many organizations, and sorting through it all takes a huge amount of CONCENTRATED work. When we got our website up in December, it was by pulling all-nighters and wrecking ourselves - and that website captures only a tiny slice of what we set out to cover. We have set up the website so anyone can contribute, but although many people have used it and praised our project, we don't get anything along these lines. We have tried to use volunteers on a casual basis, but they simply get nothing done - you need to spend hours and hours, and there's no way to divide that into minutes. It isn't like Wikipedia where each person can correct something minor or add a fact. The most important value-added is looking ACROSS organizations, which takes concentrated time.
2. Figuring out what an organization does and what the evidence is that it works is hard, not just in terms of time but in terms of getting them to answer questions. It takes back-and-forth and it takes *leverage* because most charities are reluctant to embrace transparency.
Therefore, I think what is needed is not a tool for individual donors, but a transparent grantmaker: an organization with its own staff and funds that makes large grants, does difficult due diligence, and then (this is the key) shares it all with everyone and allows anyone to comment. Speaking with people in the sector has reinforced the feeling that grantmaking foundations are positioned to evaluate charities in ways that an army of part-time volunteers never could be. And the problem with the current ones is that they don't share what they've found.
That's my impression. Thoughts?
Posted by: Holden | April 06, 2007 at 03:05 PM