Building the Project Length Calculator: Q&A
Recently the Def Method team launched a Project Length Calculator tool on the Def Method website. In this article we’ll connect with the team that built the tool to see how they overcame certain obstacles and found ways to build a helpful calculator.
Q: What were some of the challenges you encountered when building the Project Length Calculator?
Eric: We saw a few challenges associated with creating the Project Length Calculator: 1) Estimates are always hard to give, even in the context of a fluid conversation/negotiation and a tool like this concretizes a number/range that should be very flexible. 2) How do we decide what numbers to use? Based on prior projects? Based on our current workforce and their skills/speed? Do we have one or two people come up with it or make averages of a poll of all devs in the shop? 3) What does a feature mean? These are really more like feature categories, the details of which we’re simply assuming. Each product will use a certain feature differently and that unique use case might be wildly different in terms of hours from “vanilla” imagination of what a certain feature might mean.
Q: Focusing on your first point, why are estimates hard to give?
Josh: Well this gets at the fundamental problem with such a tool: every product is different. Sure there are similarities across different products and general feature-sets which we tried to identify. But every product is it’s own thing. As consultants, we’re more like old-school tailors fashioning a specific suit to a specific body frame, rather than a big factory like Ralph Lauren that pumps out tons of the same thing.
Q: So then how did you come up with the numbers to use for the estimates? How much work/research did you put into this?
Josh: There was a lot of thought put into giving worthwhile estimates but we never attempted to make it really “scientific”. We didn’t crunch a ton of numbers to make this thing. And this was partially a value proposition for Def Method internally (how much value is it worth for us to invest in this product - the Project Length Calculator, compared to the value in public good).
Q: How do you make the calculator accurate and worthwhile, but with limited data analysis and statistical intelligence?
Eric: I think it’s always a balancing act and trying to constantly make “most bang for buck” decisions based, usually, on our gut feelings, in terms of how scientific vs random to make our decisions.
Josh: yeah, right it’s always a balancing act. I mean, it’s not like we didn’t use anything to inform the numbers we chose. We used our own experience to inform the process and sent polls out to the rest of our team to determine A) what feature sets are generally requested B) basically how long does it take to implement those features.
So with that data, having done a bit of quantitative research from a highly informed survey base, we made our initial estimates.
Eric: I’m a big fan of imagining a diminishing returns curve and trying to think where on the curve we are. In my opinion, we have yet to reach a point of serious diminishing returns, meaning that every additional hour spent researching and honing numbers at this point is a well-spent hour (to a point). Buuuuut, this is only considering the product and not the overall value of the product in the context of the company as a whole. So, for example, if PLC is just on the face of it, not useful to the end user, then it might not be beneficial to spend more hours making the predictions more accurate (the inverse is also true, like if the product is incredibly valuable, that pushes out our diminishing returns curve).
Josh: Yeah, interesting point about the diminishing returns on investment in a product that hasn’t proved its worth yet. That was definitely a consideration in our development process of the PLC. We didn’t want to over invest in analysis before we saw that there was value that could be delivered to users (and in-turn our company Def Method). But this is something anyone who is developing a product must consider as they’re going. It is always hard and always dependent upon the product and market you are developing in.
Q: On the Project Length Calculator you offer the user the ability to select the platform(s) they plan to build their product on; how did you calculate the difference in development time for each platform?
Josh: Right. Good question! This had to be the trickiest portion of the product build and the one that dealt with the most complexity. However, once again, we took a measured approach. We started by splitting up the “base points” that we ascribed to every feature into separate “frontend” and “backend” components. So for instance, let’s say we initially valued the feature of “Commerce” at 10 points, then we might (purely hypothetically) split that up to 4 points on the frontend and 6 on the back. Then we went off of a general assumption that mobile products tend to be more expensive than pure web products. Mobile products require an expertise that is generally harder to find than web. Statically typed languages, compilation time, platform specific knowledge, and packaging the app are all reasons I can point to as to why mobile development tends to be “harder” or take longer than web. So going off that, for a mobile platform, we added a factor of 1.25 to the base frontend points. However, the points for any feature’s backend would not be affected by mobile vs web. Using our example from above, Commerce, on a mobile platform, we get an 11 point feature rather than 10 (4 * 1.25 = 5; 5 + 6 = 11).
Q: Do these estimates change when I select more than one platform?
Josh: Yeah, so more trickiness and complexity here. Every feature pretty much requires some frontend component and some backend component. As mentioned above, when considering development for this “platform” picker, we began by splitting frontend and backend points. So let’s say we have 1 feature that we want both on web and 1 mobile platform. For the frontend that feature would get the base points for the web plus the base points of the web times the factor 1.25. We made the basic deduction to guide our development that while each platform needs its own frontend, 1 backend can do almost all the work for multiple platforms. We didn’t go exactly with that since usually some doctoring or tweaking is required to ready an API for a new platform but generally not much. So we came up with the algorithm that each backend after the first gets a factor of 0.25 times the base points added. So going back to our example from above, Commerce, on 1 web and 1 mobile platform we get 16.5 total (1 web = 4f + 6b = 10; 1 mobile = 5f + 1.5b; 10web + 6.5mobile = 16.5)
Q: Earlier you mention that one of the challenges was deciding the estimates based on who is working on the project. Are these estimates closely tied with a certain developer level?
Eric: This question gets to one way in which Def Method is a special company. Namely, we prefer to work in pairs. Therefore, we might experience some sort of “leveling out” of the skills that we as a company (meaning, the two or more individuals who might be on your project) bring. What this means is that our estimate, in terms of who is working on the project, is more generalized over the teamwork of two of our engineers, who are inevitably going to be at different levels with regard to different aspects of software engineering. The bottom line is that the estimate will not be subject to wild variation depending on which engineers you get. This adds a comforting level of stability and reliability over a generated number that, at the end of the day, comes down to the work of flesh-and-blood people.
If you would like to ask the developers of the PLC further questions please email us at firstname.lastname@example.org