The 12-month cycle and shipping software with known bugs

The 12-month cycle and shipping software with known bugs

In a recent blog post, Deelip Menezes appears to be shocked by the very idea that a particular CAD company (no, not Autodesk) would ship software that contains known bugs. I thought he was joking, because he’s surely aware that practically all software companies with highly complex products release software with known bugs. As Deelip points out, those companies with 12-month cycles are particularly prone to doing this. There is no possible way any company can release something as complex as a CAD application within a fixed 12-month cycle without it containing dozens* of known bugs (because there isn’t time to fix them after discovery) and dozens* of unknown ones (because of insufficient Beta testing time).

Reading Deelip’s post and subsequent comments more carefully, it becomes clear that he doesn’t mean what a casual glance might lead you to believe he means. Deelip makes a specific distinction between “bugs” and “known issues”. He states that if a bug is discovered and the software is then adjusted such that it does not abort the software in a badly-behaved way, and this is then documented, then the bug ceases to be a bug and becomes a “known issue”.

I disagree. Bugs can cause crashes or not; they can cause “nice” crashes or not; they can be known about prior to release or not; they can be documented internally or not; they can be documented publicly or not. As far as I’m concerned, if the software doesn’t act “as designed” or “as intended”, then that’s a bug. Here’s what Wikipedia has to say, and I concur:

A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect or unexpected result).

That doesn’t mean that software that is “as designed” (free of bugs) is free of defects. Defects are things that make the software work in a way other than “as it should”. They can be bugs, design errors or omissions, performance problems, user interface logic failures, API holes, feature changes or removals with unintended undesirable consequences, and so on. Unfortunately, defining “as it should” isn’t a precise science. You can’t just compare the software to the documentation and say that the differences are defects. The documentation could be faulty or incomplete, or it could perfectly describe the deeply flawed way in which the software works.

While I disagree with Deelip’s definition of bugs, I couldn’t agree more with a more important point he makes in his blog post. That point is of a fixed 12-month cycle being the root cause of a plethora of bugs/issues/whatever making it into shipping software, and this being an unacceptable situation. This is a view I expressed in Cadalyst before I started participating in Autodesk’s sadly defunct MyFeedback program, and it’s a view I hold even more strongly today.

In conclusion, I would have to say that the fixed yearly release schedule is not good for AutoCAD. It is good for Autodesk, certainly in the short term, but that’s not at all the same thing as being good for AutoCAD or its users.

I’m not alone in thinking this. The polls I’ve run on this subject, discussions with many individuals on-line and in person, and many comments here and elsewhere, indicate that a dislike of the 12-month cycle is the majority viewpoint. For example, when asked the question, “Do you think the 12-month release cycle is harming the quality of AutoCAD and its variants?”, 85% of poll respondents here answered “Definitely” or “Probably”. In another poll, 71% of respondents indicated a preference for AutoCAD release cycles of 24 months or greater.

Somebody please tell me I’m wrong here. Somebody tell me that I’ve misread things, that customers really think the 12-month cycle is great, and that it’s not actually harmful for the product. Anyone?

* Or hundreds. Or thousands.

9 Comments

  1. I wouldn’t argue that a 12 month release cycle is good for the program, I also wouldn’t argue against the idea that customers believe that a 24 month cycle would be better.

    But here’s an interesting idea we don’t hear argued much when this topic is brought up: WHY does Autodesk put the software out every 12 months? Because customers DEMAND it. Now you might argue “customers don’t want it, they feel pressured into upgrading software by Autodesk and feel forced into the Subscription program to get these upgrades and support at a discount.”

    Before we go on, lets step back a bit and look at two different pictures. First the design industry. The design industry is a community that shares data between the disciplines. It’s also a highly competitive and constantly growing industry which puts a premium on staying up with technology in order to get jobs, make more profits, or to simply improve employee retention.

    Second there was Autodesk. Autodesk was a model that dominated 2D drafting, but at its core wasn’t making huge profits. The subscription program changed that and enabled Autodesk to rapidly grow into the entity it is today.. but at a price. The price is that they have to continue to generate innovation in order to keep that business model floating. What makes companies “buy in” to this? Ultimately its discounts on software that allows companies to share data between disciplines, make more profits, be in on more job opportunities, and improve employee retention.

    Do you see the link here? Autodesk is definately part of the equation, but I’d argue the INDUSTRY itself is the real problem and perhaps bigger is the free market system. This isn’t putting the blame on any company, it’s a statement of economic viability for firms, manufactures, and yes… software companies as well.

    So to answer your question/point: is it harmful to the product? In some cases, yes. But this needs to be weighed with the questions “Why are consumers DEMANDING this and FORCED to stay on subscription???”. I’d counter that this isn’t the fault of the software company.. it’s the fault of the INDUSTRY and free market system which is driving individuals to purchase software at a discounted rate in order to continue to meet their (the designers) business objectives and continue to stay viable. This past year subscription renewals have dropped dramatically, this year has been about survival and you’ve not had to worry that you competitor is ahead of you as you KNOW they are in the same boat. The result? Layoffs and cost cutting at… Autodesk.

    In the end they thrive when the INDUSTRY thrives and they contract when the industry contracts. It’s a vicious circle and a symbiotic relationship between these entities which is driving the 12 month cycle. My point: I have no problem mentioning problems, but we need to point the finger in many directions and discuss solutions to the issue as well… and viable solutions seem to be few if not non-existent.

  2. Deelip, I’ve responded to your post on your own blog.

    Brian, thank you for your thoughtful comments, but I’m not following your logic somewhere. I don’t see how you’re supporting your suggestion that customers are demanding a 12-month cycle.

  3. As the majority of companies these days are looking to find savings in their operational costs. Being forced to look at alternatives to common practice may for some bring unexpected benefits. It’s the law of unforeseen circumstances.
    So for all those paying year in year out high prices for their CAD software ‘support’, some will inevitably let the ‘support’ drop. And when in a few year’s time they wish to upgrade and are told that Auto Desk now considers them new customers and therefore want top dollar, they may just develop the courage to look around at other packages that will fulfill their requirements. And they may just find that there is a better solution to their requirements at a far cheaper price out there.

  4. Thanks for the reply Steve and sorry for not getting back around to that:

    They are demanding the 12 month cycle because of the subscription program. If you pay a yearly fee then you expect a return on your investment. In reality this is a flawed view, Autodesk could simply not release a product at all that year and be well within their rights to do so. But of course that doesn’t make for good customer relations and imagine trying to generate a subscription renewal (perhaps $600+ per seat) for software advances that may never come at all. Autodesk (and their reseller partners) wouldn’t be able to sell it. Neither would charging double for the subscription program and extending the term of it for 2 years instead of the current one. Don’t get me wrong, the client would ultimately get a better product, but they wouldn’t pay for it on a bi-yearly basis… they would feel they were getting ripped off every other year. So the customer (due to the subscription program) is demanding a new release each year even if the biggest benefit of the subscription program is the financial savings long-term of being on it… customers tend to forget that when they don’t see a steady flow of tools (may they be good or bad, used or not, ultimately they see feel they are paying for value in those tools).

  5. Patrick Weaver

    Brian, I disagree about customers demanding anything by going on subscritpion. Autodesk has simply made this the cheapest way to go for many users that need to stay on the upgrade path (even at 2-3 year intervals). Autodesk has further pushed this model with the retiring of older relases and increased upgrade prices. This is all about a more consistent revenue stream and nothing to do with serving the customer. The only side benefit to the customer is that new features would come in smaller amounts, reducing the learning curve and thus implementation into their workflow. Autodesk would still need new features regardless of the sales interval in order to continue to sell the product and retain/add customers (and stay in business).

    You’re right about a longer-interval in the subscription program. It just wouldn’t sell. There are users still in R14 (it met all their needs), and I bet some of them would still be in R12 if they hadn’t had to move into Windows.

  6. Wouldn’t subscription customers question their annual subscription budget if updates didn’t come out every year? Do AE firms put new versions immediately into production or do they review them before and wait for bug corrections?

    Don’t get me wrong, I don’t condone shipping products with bugs or defend this practice. I am not sure how Autodesk’s (we’re not talking about them?) development cycle is set up, but certainly a product can successfully have a 12 month development cycle even if some feature take 18 months to develop. Those features that are not complete would not ship.

  7. I would agree with Brian. Why do we (in our industry) buy AutoCAD? We have to to compete and to be able to use and share the data we generate. hwy do we have to buy it on subscription? So we can afford it. If subscription wasn’t available, then fewer people would update. I (or the comapnies I worked for) used to only update AutoCAD every few years (sometimes 2 or even 4). The releases were typically more spread apart in release dates, and there was no subscription incentive. Now, they are coming every year and subscription means I can get the new version every year for about the same (or close to) price as a full update every third or fourth release. Take away subscription and we wouldn’t update ever year. take away the yearly updates, and we wouldn’t join the subscription plan. Autodesk has a firm grip on the industry, until something changes the way the game is played.

  8. JG Gerth

    I’m both a user and paying customer of AutoCAD based products, and am not ‘demanding’ a 12 month release cycle. The very statment that ‘WHY does Autodesk put the software out every 12 months? Because customers DEMAND it.’ is a non-starter for several reasons.

    First of all, the customers do not speak with one voice. Some may want annual updates, some may want new features, others want support and continued functionality, some want interoperability, some want transparency, but ALL of them want stability. We users and customers are not running CAD because it’s fun – we are using CAD as a tool to help our businesses survive and accomplish our drafting and design goals. Bugs, known issues, memory leaks, crashes, unexpected behaviors — all inevitable side effects of the 12 month release cycle — hinder our ability to accomplish our jobs, pay our bills, and feed our families.

    Secondly, Autodesk senior executives don’t listen to ‘demands’ from users or customers. Stockholders maybe they’ll listen to, but the last demand from users that I saw responded to by senior people in Adesk was the Dimensioning capabilities pulled together by John Walker back in the dawn of time.

    Now there’s another point that’s being missed about subscription. It appears from the comments here and elsewhere that the perceived value of subscription is solely limited to getting the next version pre-paid instead of penalized. That’s our problem as well as Autodesks.

    Ours because we are on the treadmill of endless incremental updates, file format changes, incompatible versions, and in many organizations the need to keep everyone in lockstep. Autodesk’s because they have failed to create a value proposition to the subscription program other than annual release of new versions (and the problems with the rewritten, untested software).

    And this is fixable! This is something Autodesk can do. They can improve the value of subscription to users, customers, and encourage subscription signups by increasing the value of the subscription program, improving support, adding more and better tutorials, moving AU content solely to the subscription site, add perks, improve the Express Tools as a subscription only benefit, provide better backwards compatability, and post updates to earlier versions of the software to enable reading and working with newer versions.

    That last may seem counter-intuitive, but many many organizations that are on subscription continue to run earlier version for a multitude of reasons. The question they are facing is if we are not going to deploy the bleeding edge, and that’s the sole benefit of subscription, why the heck should we stay on it? How many DVDs of 2k7, 2k8, 2k9, and 2k10 are sitting on shelves because an organization is not going to implement them? based on my personal experience in several ENR top 100 firms, the answer is ‘lots’.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.