Long time readers of this blog will know about my love of the number of inquiries filter. I have talked about it a great deal but I have never explained in detail why I use this as my primary filter when choosing loans.
Today, I will give you some analysis that will back up my claims about this filter. But first let’s explain what I mean by number of inquiries. This is something that is recorded on everyone’s credit report and is used by Lending Club and Prosper in their underwriting. When you apply for a credit card, a car loan, a home loan or almost any other kind of loan your credit report gets marked with an inquiry. Any loan application where they do what is called a hard pull of your credit will stay on your credit report for six months.
What is interesting to me is that when you filter loans on the number of inquiries you find that the more inquiries a borrower has the more likely they are to default and the lower your expected ROI is going to be. This rings true for both Lending Club and Prosper borrowers. Here are the numbers for Lending Club. Keep in mind I am only looking at data from July 2009 onwards and, as always, I used data from Lendstats to create these charts.
Lending Club Analysis
| All Loans ROI | All Loans Default Rate | Grade D-G ROI | Grade D-G Default Rate | |
|---|---|---|---|---|
| Inquiries = 0 | 7.84% | 1.34% | 9.21% | 2.19% |
| Inquiries <= 1 | 7.17% | 1.59% | 8.36% | 2.36% |
| Inquiries <= 2 | 7.00% | 1.71% | 7.99% | 2.54% |
| Inquiries <= 3 | 6.73% | 1.85% | 7.63% | 2.66% |
| Inquiries >= 4 | 3.33% | 4.51% | N/A | N/A |
If you choose all loan grades on Lending Club that have zero inquiries on their credit report, according to Lendstats, you can expect a 7.84% ROI (as of this writing) on your money. Then for each additional inquiry the ROI goes down and the default rate goes up. This trend gets even more noticeable when inquiries are greater than three.
The number of credit inquiries has an even bigger difference when you look the lower grade (higher risk) loans. When taking grades D-G only on Lending Club you can see that the range is from 9.21% down to 7.63%. I didn’t include the numbers for Grades D-G for inquiries >= 4 because there were only 53 loans on the entire database so it wasn’t a large enough sample to give meaningful data.
Prosper Analysis
| All loans ROI | All loans Default Rate | D-HR ROI | D-HR Default Rate | |
|---|---|---|---|---|
| Inquiries = 0 | 10.12% | 2.71% | 15.55% | 3.82% |
| Inquiries <= 1 | 10.06% | 3.11% | 14.70% | 4.16% |
| Inquiries <= 2 | 9.98% | 3.23% | 14.18% | 4.27% |
| Inquiries <= 3 | 9.80% | 3.42% | 13.70% | 4.51% |
| Inquiries >= 4 | 7.33% | 6.01% | 9.02% | 6.19% |
We find the same trend on Prosper (using data from July 2009 onwards) although admittedly it is not quite as marked. But for every additional inquiry that a borrower makes on their account we see the same linear progression: estimated ROI goes down and default rate goes up. And if you are investing in higher risk loans then again the difference is even more noticeable.
This Makes Logical Sense
When you think about it, this makes logical sense. If someone is out shopping for credit in several places and then they come to Lending Club or Prosper looking for a loan then they are a higher risk borrower. They may have some serious financial problems if they are shopping for a lot of credit. But if someone is looking for a loan and comes to p2p lending first then they are a better credit risk. One inquiry on a credit report doesn’t make a big difference but the numbers show that these borrowers are still a higher risk than borrowers with no inquiries at all in the past six months.
This is why I continue to make the number of inquiries a key part of my investment strategy and I encourage other investors to do the same.
[Update: several people indicated that the above charts gave an unclear indication of the differences between number of inquiries because I wasn’t isolating each inquiry number in my analysis. So in order to give a complete picture for investors I have redone the charts below with the number of inquiries isolated (as in inquiries = 1 instead of inquiries <= 1). The same trends still apply although it is not quite as linear. Please note that the charts below were created on a different day to the original charts which is why the numbers for inquiries = 0 and inquiries >= 4 are slightly different.]
Lending Club Analysis 2
| All Loans ROI | All Loans Default Rate | Grade D-G ROI | Grade D-G Default Rate | |
|---|---|---|---|---|
| Inquiries = 0 | 7.85% | 1.36% | 9.31% | 2.22% |
| Inquiries = 1 | 5.99% | 2.05% | 7.00% | 2.36% |
| Inquiries = 2 | 6.14% | 2.38% | 6.47% | 3.34% |
| Inquiries = 3 | 3.77% | 3.60% | 4.58% | 3.72% |
| Inquiries >= 4 | 3.38% | 4.51% | N/A | N/A |
Prosper Analysis 2
| All loans ROI | All loans Default Rate | D-HR ROI | D-HR Default Rate | |
|---|---|---|---|---|
| Inquiries = 0 | 10.06% | 2.72% | 15.32% | 3.86% |
| Inquiries = 1 | 9.88% | 3.87% | 13.52% | 4.60% |
| Inquiries = 2 | 9.47% | 3.94% | 11.80% | 4.67% |
| Inquiries = 3 | 6.40% | 6.12% | 8.45% | 6.95% |
| Inquiries >= 4 | 7.22% | 5.94% | 8.87% | 6.10% |
First off, I love Lendstats.
The “problem” with the filters is that there are too many levers to play with.
So far the depth of my analysis has been “If I change this, does ROI go up or down?”. So while I have played with each variable to see what happens, that is not good enough.
What is needed is a multivariant analysis or Monte Carlo simulation to find the best combination.
Anyone done anything like this?
why do you have the top columns as being =? It is very deceptive in making 4 look much worse than the others, when this is not actually true. Having 4 inquiries is nearly the same as having 3, but you would never recognize that from your charts.
“>=3” gives a result of 3.69% for all loans ROI which is nearly as bad as >=4.
“<=4" gives a result of 6.73% for all loans ROI which is identical to "<=3".
Statistics should all cover similarly directioned ranges in order to avoid this deception. Essentially you are making the 3 inquiries look much better than they are by grouping them with the zeros and making the 4's look much worse by not grouping them with zeros.
Great job on the analysis! Indeed there seems to be a strong correlation between the maximum # of inquiries and Default rate.
Do you think it would be possible to make another chart with the row labels: inquiries =1, inquiries =2, inquiries = 3, etc.? Or is the sample size not big enough?
That way, it becomes incredibly clear that the higher the # of inquiries a borrower has, the higher the chance of default.
@ Brian B
Indeed by including people with 0,1,2 in the </= 3 catagory you are giving alot of weight to the 0,1,&2 in your overal ROI calculation.
The stats look differnt if you look at =0 , = 1 , = 2 , = 3 , =4 then if you look at 0, </=1 , </=2 , /= 4
@Charlie/Brian/P2PNoob, Thanks for your comments. Well I certainly don’t want to be deceptive in anything I do, so I have redone the tables as per your requests above. The reason I did it this way was because this is the way I assume most investors choose loans. People rarely will say I want to invest in Inquiries = 2 they typically say they will invest in loans up to 2, which is Inquiries <=2. So I felt be presenting the tables in this way it would give investors a better comparison they can use. But I can see that others disagree with that and so there are two sets of tables now in this post. I believe the same trends still apply although it is not completely linear any more. I am certainly not a statistician (I took one statistics class in college 25 years ago) so I am happy to stand corrected on some of this. And @Charlie, I am sorry but I have no idea what a Monte Carlo simulation is.
@Peter………… Considering that last week you actually cited the figure 500 as an example when discussing the Law Of Large Numbers, I’m guessing that your former Stat prof would retroactively lower your grade if he could ! 🙂
@Dan, I am sure he wouldn’t have been surprised. It is one area I probably should educate myself a little more.
@Dan
Thanks for making the new table!
Interesting that the difference in risk of default from 1-inquiry and 0-inquiry borrowers on Lending Club is only .14%. But it’s much greater on Prosper at .74%. (yet, by looking at the ROI differences, Prosper must really compensate well for the risk.)
@Peter I meant. Sorry, the last message I read was Dan’s and accidentally typed it
@P2PNoob, I think the D-HR group on Prosper is a higher risk that the D-G on Lending Club. Look at the credit scores of these loans and you will notice that Prosper has a lower average than Lending Club for these lower grade loans. But as you say, investors are rewarded for taking this extra risk.
I agree with Charlie H about the need for multivariate analysis and Monte Carlo simulations. These can be used to better assess financial risk. I strongly suspect the institutional investors who are involved P2P lending are already doing this.
For those who are interested, here are some good sites discussing the Monte Carlo method:
http://www.palisade.com/risk/monte_carlo_simulation.asp
http://en.wikipedia.org/wiki/Monte_Carlo_method
For those that want to brush up on their statistics, check out:
http://www.khanacademy.org
Any starving graduate students in statistics or finance looking for a great thesis project? 😉
@Lee, Thanks for the resources, I will check them out. And I would love to have a grad student work on some of these p2p lending numbers, send them my way!
@Lee B
I have a Medical stats back ground and am familure with MVA and MC from that perspective.
I had no idea that they were being applied to finance problems, but it makes sense since the there are so many degrees of freedom in finance problems.
Lendingstats has so much information to data mine. Doing single varriant analysis (If I change this varriable does ROI go up or down?) is useful, but it only scratches the surface.
Charlie H and Lee B,
I am the creator of lendstats and I’ve added a new feature which you may find very useful. Now when you change a variable, you can generate a complete analysis report that shows the ROI’s of most criterion by segments. So not only will you just see how changing one variable affects overall ROI, you can also see how it affects most other segmented criterion. This is great for noticing what the interaction effects are. Right now I have almost 200 different segments for which you can see ROI’s and I will be adding a few others.
For example this link will show you the analysis report (or as I call it at lendstats, the “Complete Performance Breakdown”).
http://www.lendstats.com/loansearch/lc/lcloanfilter.php?sho1=2
It’s only ready for LendingClub data, I’m still working on the Prosper version (but there already is a lite version for the Prosper data).
@Charlie, Well I know who to go to know if someone asks me the tough statistics questions. Feel free to do some experiments of your own.
@Ken, This is a great new feature, thanks. Now, when I make one change in my filters I can see the ripple effect on everything – makes for a much more complete picture. Keep up the great work.