How to guarantee the success of a project

I've just finished reading a thought-provoking article on about scope definition and the heavy price of ignoring this critical phase of a project.
It brought to mind the old chestnut about stopping to smell the roses. It seems to me that stopping to smell the roses is more than a trite piece of advice on how to live a good life: stopping to smell the roses is a key factor in the success of a project.

Many of us, when confronted with a project that needs to get done, are anxious to complete the project and eager to get started. By the time many clients approach me, they've been debating and wringing their hands for some time. Thus, there's no time to lose, and we must start the project right away. However, when asked to define the problem and to describe the constraints under which the problem must be solved, many are unwilling to take the time to do so. After all, so much time has already passed between the time the client acknowledged the problem and conceded the budget to resolve it.

And yet this unwillingness to fully describe the problem is precisely the cause of project failure: the failure of a vendor partner to deliver the right solution, the failure of the project being completed on time and within budget. Whose FAULT is it?

Project failures are more often than not the result of irresponsibility on the part of both the client and the vendor partner. They are the result of failing to stop and think before an action is taken. They are the result of the mindset of "if you start building it, the right answer (i.e. the scope) will materialize or can be managed".

Once a project starts, it becomes a stressful juggling act for all involved: the client, the account manager, and the project manager. The account manager struggles to manage scope creep that erodes profit margin and client satisfaction. The project manager struggles to nail down a problem long enough for a solution to be built before new requirements are thrown into the pit to jeopardize budgets and timelines. The client struggles with a superior who cannot understand why the product they've paid money to get produced isn't meeting the company's needs and is now delayed, just to add insult to injury.

So why do so many agencies put themselves in such situations, why do we capitulate to poorly defined scope requirements when agencies are the ones who end up paying the price of failure? It's simple enough to say it's the fear of losing the project to a competitor; or to say that in business (and in life), nothing ventured is nothing gained. It's hard for me to accept this because a real alternative exists that practically guarantees project success and all-round happy project participants.
From a client's perspective, any vendor partner who leaps head first into a project without asking many detailed questions is a risk factor. Such vendor partners might be the lowest bidder at project initiation, but the math might not work out that way when the project fails.

Are methodologies scalable?

Requirements gathering is one of those steps in any project that can't be missed. Whether it's conducted vigorously and methodically, haphazardly and unconscious, it's done.

It doesn't matter whether you're ordering a hamburger at Harvey's or a web site build: the person taking your order needs to gather your requirements.
When you go to Harvey's, you don't just say you want a hamburger; the person taking your order will want to know if you want a classic or whatever, the person making your burger will want to know what kind of toppings you want. You could be remiss and say "just give me the toppings most people get"; not a problem, unless you really detest ketchup, which is what most people get on their burger.

Recently, I was discussing with a colleague the issue of requirements gathering methologies. In our experience, SMB's and micro-businesses, in particular, tend to work within very short temporal cycles. They can also have pretty short fuses and regard methodical questioning and any information probing to be, at best, extemporizing tactics and, at worst, unresponsiveness. How often have you been challenged with, "Why do you need to know so much? Why are you asking all these questions? I just want a ballpark, an estimate. I'm just a small business and don't have the time to be talking about this. I just need to get it done."

So we came to the tentative conclusion that enterprise-level business methologies don't necessarily scale down to the micro-business and even SMB (small to medium businesses) level. Is this really true? Do we need to develop a completely different style of requirements gathering for different classes of clients?