The Software Testing TimelineClick on the image to see a larger version.
The Software Testing Timeline by Joris Meerts is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
1822 Difference engine (Babbage)
The English inventor Charles Babbage starts working on a prototype of his first difference engine. The difference engine is made to compute values of polynomial functions. Babbage's machines were among the first mechanical computers.
1843 Methods of inductive reasoning (Mill)
In the A System of Logic the British philosopher John Stuart Mill publishes five methods of inductive reasoning; the reasoning from a specific case or cases to a general rule. The fundamental principle of induction is the proposition that the course of nature is uniform.
1843 Lovelace's notes on the Analytical Engine
During a nine-month period in 1842–43, Lovelace translates Italian mathematician Luigi Menabrea's memoir on Charles Babbage's Analytical Engine. With the article, she appends a set of notes. The notes include a method for calculating a sequence of Bernoulli numbers with the Engine. Based on this work, Lovelace is now widely credited with being the first computer programmer and her method is recognised as the world's first computer program.
1878 System flaw termed as bug (Edison)
According to the The Yale Book of Quotations the American inventor Thomas Alva Edison uses the term 'bug' in a letter to Theodore Puskas to describe a flaw in a system. According to other sources the term bug was commonly used to describe systems faults in Edison's time.
1896 IBM founded
The company which becomes IBM is founded in 1896 as the Tabulating Machine Company by Herman Hollerith in Broome County, New York. It is incorporated as Computing Tabulating Recording Corporation on June 16, 1911. The company is named International Business Machines Corporation (IBM) in 1924.
1906 Pareto Principle (80/20 rule)
In his book Manual of Political Economy the Italian economist Vilfredo Pareto publishes the mathematical formula to describe the unequal distribution of wealth in his country, observing that twenty percent of the people owned eighty percent of the wealth. The formula becomes known as the Pareto principle but also, through popularization by Joseph Juran, as the 80/20 rule.
1939 Shewhart Cycle
In his second book Statistical Method from the Viewpoint of Quality Control Walter Andrew Shewhart publishes his Scientific Method of Improvement. Page 45 of this book displays the first version of the Shewhart Cycle; a three step cycle constituting a dynamic scientific process of acquiring knowledge. Walter Edwards Deming later popularizes the method as the Plan Do Check Act method (The Deming Cycle).
1944 Harvard Mark I installed
The Harvard Mark I, also known as the IBM Automatic Sequence Controlled Calculator (ASCC), is installed at Harvard and formally presented on August 7, 1944. The Mark I is a parallel synchronous calculator that can perform table lookup and the four fundamental arithmetic operations, in any specified sequence, on numbers up to 23 decimal digits in length. The machine was conceived by Howard H. Aiken in the 1930s.
1945 Von Neumann architecture
In the paper First Draft of a Report on the EDVAC John von Neumann published the description of the logical design of a computer using the stored-program concept. This is the design on which most computers nowadays are based. The paper is distributed on June 30, 1945 by Herman Goldstine.
1946 ISO founded
The International Organization for Standardization (ISO) is founded during a conference in London in 1946. ISO is born from the union of two organizations; the ISA (International Federation of the National Standardizing Associations), established in New York in 1926 and the UNSCC (United Nations Standards Coordinating Committee), established in 1944.
1949 On Checking a Large Routine (Turing)
In the conference paper On Checking a Large Routine Alan M. Turing proposes an answer to the question how one can check a routine in the sense of making sure that it is right.
1950 Turing Test (Turing)
In his 1950 paper Computing Machinery and Intelligence Alan M. Turing introduces the Turing Test; a test of a machine's ability to demonstrate intelligence.
1951 Juran's Quality Control Handbook (Juran)
In his book Juran's Quality Control Handbook Joseph M. Juran defines quality as 'fitness for use'. Juran, who is considered to be the founding father of quality management, defines three processes for the management of quality; quality planning, quality control and quality improvement.
1951 Total Quality Control (Feigenbaum)
In his famous book 'Total Quality Control' Armand Vallin Feigenbaum defines quality as a customer determination. Quality depends on the perspective of the customer. The product should satisfy the customer in both actual and expected needs. There is a company-wide responsibility for quality.
1951 First practical text on programming (Wilkes)
The book The Preparation of Programs for an Electronic Digital Computer by authors Maurice V. Wilkes, David J. Wheeler and Stanley Gill is the first practical text on computer programming. The authors draw on their experience from designing the EDSAC.
1952 IBM 701 released
The IBM 701 Defense Calculator is IBM's first production computer. It is the first of the very popular IBM 700 series. The computer contains two electrostatic storage units and an electronic analytical and control unit. 19 units are created. A unit rented for about $16,000 per month.
1956 First Fortran manual
The first Fortran manual (an automatic coding system for the IBM 704) is published by IBM in October 1956. It describes the variables, functions, expressions and control statements of the language.
1957 Program testing vs debugging (Baker)
Charles L. Baker (RAND Corporation) distinguishes program testing from debugging in his review of the book Digital Computer Programming by Dan McCracken. The review is published in the journal Mathematical Tables and Other Aids to Computation.
1957 Digital Computer Programming (McCracken)
Daniel D. McCracken's book Digital Computer Programming is perhaps the first general textbook on programming. McCracken writes the book while working for General Electric. He discusses a number of programming techniques such as program checkouts. McCracken specifically advises using break points in a program at which the functioning of the program may be verified.
1958 First software test team (Weinberg)
The first test team is formed by Gerald M. Weinberg, working as manager of Operating Systems Development for the Project Mercury. Project Mercury is the first human spaceflight program of the United States.
1959 Work Breakdown Structure
In their classic paper Program Evaluation and Review Technique (PERT) D.G. Malcolm, J.H. Roseboom, C.E. Clark and W. Fazar introduce their method to analyze the involved tasks in completing a given project. Although not mentioned by that name in the paper they also introduce the Work Breakdown Structure.
1959 Critical-Path Method (Kelley, Walker)
In their paper Critical-Path Planning and Scheduling James E. Kelley and Morgan R. Walker offer a planning and scheduling methodology for complex projects. They introduce the Critical-Path Method in order to create optimal direct cost schedules. The authors use project diagrams of tasks and link these tasks (jobs) based on three questions; what immediately precedes this job, what immediately follows this job and what can be concurrent with this job?
1961 Computer Programming Fundamentals (Leeds, Weinberg)
The book Computer Programming Fundamentals by Gerald Weinberg and Herbert Leeds contains a chapter on software testing. Weinberg and Leeds argue that testing should prove the adaptability of a computer program instead of its ability to process information.
1961 Decision table (Grad)
In the paper Tabular Form in Decision Logic, published in Datamation in July 1961, Burton Grad, working at IBM, introduces decision tables.
1963 Flowchart symbols standardized (Rossheim)
In his paper Report on proposed American standard flowchart symbols for information processing Robert J. Rossheim describes the standard symbols to be used for flowcharting. Rossheim publishes his work for the American Standards Association, later ANSI.
1964 First Basic program runs
Mathematicians John G. Kemeny and Thomas E. Kurtz run the first Basic program at Dartmouth College. Kemeny and Kurtz write the language to aid the teaching of computer programming. Basic is an acronym for Beginner's All-Purpose Symbolic Instruction Code.
1967 Evaluation of the Functional Testing of Control Programs
In the IBM white paper Evaluation of the Functional Testing of Control Programs William Elmendorf calls for a disciplined approach to software testing.
1967 Software engineering introduced
The term 'software engineering' is coined by a NATO study group that recommends a conference to discuss 'the problems of software'. The report from the ensuing 1968 conference, which was sponsored by the NATO Science Committee and took place in Garmish, Germany, is titled Software Engineering. Brian Randell and Peter Naur point out in the introduction to their edition of the proceedings, "The phrase 'software engineering' was deliberately chosen as being provocative, in implying the need for software manufacture to be [based] on the types of theoretical foundations and practical disciplines[,] that are traditional in the established branches of engineering.
1968 NATO report mentions Software Quality Assurance
During the Software Engineering conference sponsored by the NATO Science Committee (7th to 11th October 1968) among other things quality assurance for software production is one of the topics. The report of the conference includes the working paper Checklist for planning software system production by Robert W. Bemer. This paper contains a chapter on quality assurance. One of the questions in the checklist is 'Is the product tested to ensure that it is the most useful for the customer in addition to matching functional specifications?'.
1968 Structured programming (Dijkstra)
In his letter Go To Statement Considered Harmful published in the March 1968 Communications of the ACM, Edsger Dijkstra criticizes the excessive use of the GOTO statement in programming languages. Dijkstra considers the usage harmful because it complicates the task of analyzing and verifying the correctness of programs. The letter is usually cited as the beginning of the structured programming debate
1968 Intel founded
Intel is founded by Gordon E. Moore and Robert Noyce. In the beginning the company's primary products are static random access memory (SRAM) chips. Intel releases the first commercially available microprocessor (Intel 4004) in 1971.
1968 Software crisis (Bauer)
The term 'software crisis' is coined by Friedrich Ludwig Bauer at the first NATO Software Engineering Conference in 1968 in Garmisch-Partenkirchen, Germany. It refers to the difficulty of writing correct, understandable, and verifiable computer programs.
1968 Conway's Law
In his 1968 paper How Do Committees Invent? Melvin E. Conway publishes the thesis that "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.". The thesis later becomes known as Conway's Law.
1969 Testing shows the presence, not the absence of bugs
Edsger Dijkstra's famous quote was reportedly first spoken on a conference by the NATO Science Committee, Rome, Italy, 27–31 October 1969.
1969 Data structure diagrams (Bachman)
In the article Data Structure Diagrams Charles W. Bachman, working for General Electric, introduces Data Structure Diagrams. It is a graphic technique that is used to model entities and their relationships.
1970 Waterfall model published (Royce)
Winston Royce describes a waterfall model in the paper Managing the Development of Large Software Systems, presented to IEEE WESCON in 1970. The waterfall model is a sequential software development process. In his paper Royce also mentions the shortcomings of the waterfall and warns against using the method.
1970 Relational model published (Codd)
In his paper A Relational Model of Data for Large Shared Data Banks Edgar F. Codd, working for IBM, publishes the relational model for data. His theory paves the way for relational databases.
1971 Mutation testing (Lipton)
In a class term paper titled Fault Diagnosis of Computer Programs Richard Lipton proposed the initial concepts of mutation. Mutation testing is a methodology for unit testing in which small parts of the code are changed. This is done, for example, in order to test the quality of the unit tests.
1972 Systems decomposition (Parnas)
In his famous paper On the Criteria to Be Used in Decomposing Systems into Modules David Lorge Parnas discusses the benefits of modular programming.
1972 Proof of correctness as guidance (Dijkstra)
In his ACM Turing Lecture titled The Humble Programmer Edsger W. Dijkstra states that "One should not first make the program and then prove its correctness, because then the requirement of providing the proof would only increase the poor programmer's burden." It is one of his arguments in support of the technical feasibility of the change required in software programming.
1973 Program Test Methods (Hetzel)
The Chapel Hill Symposium, organized by the University of North Carolina and held on June 21-23 1972, leads to publication of the book Program Test Methods edited by William Hetzel. The book contains the edited papers of the symposium as well as a large annotated bibliography. The book focuses on the problems in testing and validation.
1973 Cause-effect graph created (Elmendorf)
In the paper Cause-Effect Graphs in Functional Testing, published for IBM, William Elmendorf publishes the cause-effect graph.
1973 Compuware founded
Compuware Corporation is established in Detroit by Peter Karmanos, Jr., Thomas Thewes and Allen B. Cutting. The company focuses on the publication of testing tools and engineering aids.
1973 Stages of growth model (Nolan)
In his paper Managing the computer resource: a stage hypothesis Richard L. Nolan introduces his Stages of growth model. The model describes the stages a company goes through when introducing information technology (computers) into its business process. Initially there are four stages. Two additional stages are added in 1979.
1973 Triangle problem (Gruenberger)
In his paper Program Testing: the Historical Perspective - based on the talk held during the Chapel Hill Symposium - Fred Gruenberger introduces the classic triangle problem as a testing idea. Gruenberger takes the problem from the book Computers and Society by Richard Hamming (1972).
1973 Nassi-Shneiderman diagram
In their paper Flowchart Techniques for Structured Programming Isaac Nassi and Ben Shneiderman publish their flowchart for the top-down decomposition of programs.
1974 Data flow diagram (Constantine)
In their paper Structured Design, Larry Constantine, Glenford Myers and Wayne Stevens introduce the Data flow diagram.
1974 Walk through (Waldstein)
In a technical report to IBM, entitled The Walk-Thru--A Method of Specification, N.S. Waldstein introduces the concept of walk throughs.
1975 Microsoft founded
Microsoft is founded by Bill Gates and Paul Allen on April 4, 1975 in Albuquerque. It is to become one of the largest software companies in the world. The company employed over 10,000 testers in 2009.
1975 Laws of Unreliability (Gilb)
Tom Gilb publishes his Laws of Unreliabilty in the computer magazine Datamation (March issue, 1975). Gilb is one of the first to define system and software reliability and the relationship between human error and system error.
1975 Toward a Theory of Test Data Selection (Goodenough, Gerhart)
The paper by John B. Goodenough and Susan L. Gerhart discusses formal proof methods and the limitations of structure-based testing. It also outlines the use of decision tables.
1975 Brooks' law
In the book The Mythical Man-Month: Essays on Software Engineering Fred Brooks presents the central theme that "adding manpower to a late software project makes it later". This idea is known as Brooks' law, and is presented along with the second-system effect and advocacy of prototyping. Brooks draws from his experience as a manager of the development of IBM OS/360.
1975 Introduction to General Systems Thinking (Weinberg)
In his book An Introduction to General Systems Thinking Gerald M. Weinberg offers a clear view on systems theory. According to Weinberg it is an attempt "to teach an approach to thinking when the labels are missing, or misleading." Understanding the limitations of our tools is a recurring theme in the book.
1975 Altair 8800 introduced
The Altair 8800 is introduced by Micro Instrumentation and Telemetry Systems (MITS) in 1975. It is a microcomputer based on the Intel 8080 processor. The computer sells for $439 in kit form and $621 assembled. It is sold by mail order through advertisements in Popular Electronics, Radio-Electronics and other hobbyist magazines. It runs Altair Basic.
1976 Design and Code inspections (Fagan)
In his paper for IBM Michael Fagan proposes the use of code inspections in order to reduce the cost of rework.
1976 Apple founded
Apple is established in Cupertino, California by Steve Jobs, Steve Wozniak, and Ronald Wayne.
1976 Cyclomatic Complexity (McCabe)
Thomas J. McCabe introduces cyclomatic complexity as a software metric for the complexity of a program in his IEEE paper A Complexity Measure. McCabe also introduces basic path testing as a white box test technique.
1976 Software Reliability: Principles and Practices (Myers)
In his book Software Reliability: Principles and Practices Glenford Myers discusses software testing among other things. He mentions, for example, that 'The goal of the testers is to make the program fail'.
1976 Cost-of-change curve (Boehm)
In his paper Software Engineering, published in the December 1976 issue of IEEE Transactions , Barry Boehm publishes his cost-of-change curve. The curve essentially shows that the cost of changing the software (fixing a software defect) rises exponentially in time. Boehm uses data from his work at TRW and other sources such as GTE, IBM and Bell Laboratories.
1977 Factors in Software Quality (McCall a.o.)
In this paper for the US Air Force Jim McCall, Paul Richards and Gene Walters (General Electric) present a list of 55 quality factors.
1977 Software Metrics (Gilb)
The book Software Metrics by Tom Gilb is considered a source book for a large number of software metrics.
1977 Elements of Software Science (Halstead)
Maurice Halstead publishes his methods for the measurement of code complexitity in the book Elements of Software Science.
1977 Program Testing Techniques (Miller)
Edward F. Miller introduces the decision-to-decision path as a structural testing technique in the tutorial Program Testing Techniques, held during the first Computer Software and Applications Conference (COMPSAC).
1977 Design pattern (Alexander)
In their book A Pattern Language: Towns, Buildings, Construction Christopher authors Alexander, Sara Ishikawa and Murray Silverstein introduce the term 'design pattern'. They apply it to architecture and urban development. In software engineering, a design pattern is a general reusable solution to a commonly occurring problem in software design.
1978 Functional Program Testing (Howden)
In this paper for IEEE William Howden describes an approach to functional testing in which the design of a program is viewed as an integrated collection of functions.
1978 Laws of program evolution (Lehman)
In the paper Laws of Program Evolution - Rules and Tools for Programming Management Meir Lehman publishes the laws of program evolution. They are the law of Continuing Change, Increasing Complexity, Statistically Regular Growth, Invariant Work Rate and Incremental Growth Limit. Based on the laws Lehman suggests rules and tools for software development and maintenance. Lehman's work is based on a study done in collaboration with Laszlo Belady.
1978 State transition testing (Chow)
In his paper Testing Software Design Modeled by Finite-State Machines, Tsun S. Chow introduces testing techniques for state transition testing. Chow specifies measures for testing coverage.
1979 The Art of Software Testing (Myers)
The book The Art of Software Testing by Glenford Myers is lauded as the first book that is about software testing only. It sets the stage for 'modern' software testing. Among other things Myers introduces black box testing. He also poses the 'Triangle Program' as a testing self-assessment challenge.
1979 Quality is Free (Crosby)
In his book Philip B. Crosby describes his famous 14-step program for quality improvement. Crosby defines quality as conformance to requirements. He defines the 'Quality Maturity Grid' and classifies management attitudes toward quality into five categories. Crosby suggests that improvement happens as management 'matures' from one category to another.
1979 QA standard BS 5750 published
The British quality assurance standard BS 5750 is published by the British Standards Institution. Initially, it was used only in manufacturing industries. BS 5750 was later merged into ISO 9000.
1979 FMEA applied to software (Reifer)
In his paper "Software Failure Mode and Effects Analysis" Donald J. Reifer applies Failure Mode and Effects Analysis (FMEA) to software for the first time.
1979 Structured Analysis and System Specification (DeMarco)
The book Structured Analysis and System Specification by Tom DeMarco is a classic on structured analysis. DeMarco covers functional decomposition, data dictionary, process specification, system modeling and structured analysis for a future system. He suggests an approach that emphasizes a top-down, partitioned, graphic model of the system-to-be. He also stresses the important role of a data dictionary and the role of scaled-down specifications, or minispecs, to be written in a rigorous subset of the English language known as Structured English.
1979 Function points (Albrecht)
In his paper Measuring Application Development Productivity Allan Albrecht, working for IBM, introduces the function point in order to measure programmer productivity. Albrecht concludes from his study of productivity that a discplined process is "an essential ingredient to meaningful productivity measurement".
1981 Software Engineering Economics (Boehm)
In the landmark book Software Engineering Economics Barry W. Boehm introduces the notion that the cost of fixing a bug rises exponentially in time. He also introduces the constructive cost model (COCOMO).
1981 IBM Personal Computer goes mass market
IBM releases the personal computer (the IBM-PC or IBM 5150) that set the standard for mass market computer products.
1981 Rational Software founded
Rational Software is founded by Mike Devlin and Paul Levy, Until its acquisition by IBM in 2003 Rational produces tools to aid software engineers, among which quality management tools such as ClearQuest.
1981 What is Total Quality Control? (Ishikawa)
In his book What is Total Quality Control? The Japanese Way (first published in Japanese in 1981) Kaoru Ishikawa states that that quality is a dynamic concept as the needs, the requirements and the expectations of a customer continuously change.
1981 CRUD analysis (Martin)
In their report entitled Information Engineering, for the Savant Institute, James Martin and Clive Finkelstein introduce the term CRUD (Create, Retrieve, Update, Delete) analysis, as a method for analyzing design.
1982 Rethinking Systems Analysis and Design (Weinberg)
In this book Gerald Weinberg describes iterative development and testing.
1982 First PNSQC conference
The first Pacific Northwest Software Quality Conference (PNSQC) is held in Portland, Oregon. The goal of the non-profit Oregon corporation is to enable knowledge exchange to produce higher quality software.
1982 SQS founded in Germany
The German company Software Quality Systems (SQS) is founded Heinz Bons and Rudolf van Megen. It is one of the leading software testing organisations in Europe.
1982 Out of the Crisis (Deming)
In his book Out of the crisis : Quality, Productivity and Competitive Position William Edwards Deming proposes that quality must be defined in terms of customer satisfaction. Deming lists fourteen key principles for management that should aid the implementation of his view of quality.
1982 Life Cycle Concept Considered Harmful (McCracken)
In their paper Life Cycle Concept Considered Harmful Daniel McCracken and Michael A. Jackson criticize the waterfall way of software development. Among other things they state that "System requirements can never be stated fully in advance, not even in principle, because the user doesn`t know them in advance - not even in principle“.
1983 IEEE 829 published
The first version of the IEEE 829 Standard for Software Test Documentation is published in 1983. The standard specifies the form of a set of documents for use in eight defined stages of software testing.
1983 US federal V&V standard published
The Guideline for Lifecycle Validation, Verification, and Testing of Computer Software (a Federal Information Processing Standard (FIPS 101)) is published by the United States federal government.
1983 Psychology of Human-Computer Interaction
The book The Psychology of Human-Computer Interaction by Stuart K. Card, Thomas P. Moran, Allen Newell is considered a classic in the field of usability. It discusses, among other things, psychologically-based design methods and tools such as GOMS (Goals, Operators, Methods, and Selection rules).
1984 An Information Systems Manifesto (Martin)
In the book An Information Systems Manifesto James Martin publishes his Distribution of Defects, showing that over half of the defects have their root cause in poorly defined requirements.
1984 First testing industry conference (USPDI)
The US Professional Development Institute (USPDI) organizes the International Conference and Exposition on Testing Computer Software; the first conference for the testing industry. The conference is held in Washington DC. Dave Gelprin and Bill Hetzel are joint programme chair for the first six years. Ginger Houston-Ludlam is programme chair for the next ten years. Denis Meredith is programme chair for the remaining years. The last of these conferences is held in 2002.
1984 SEI founded
The Carnegie Mellon Software Engineering Institute (SEI) is established by the U.S. Department of Defense. In its own words "the SEI advances software engineering and related disciplines to ensure the development and operation of systems with predictable and improved cost, schedule, and quality."
1984 Theory of Constraints (Goldratt)
In his book The Goal Eliyahu M. Goldratt publishes his Theory of Constraints. It is a management philosophy that is focused on eliminating constraints, bottlenecks from a production process. Theory of Constraints is based on the premise that the rate of goal achievement is limited by at least one constraining process
1985 Microsoft releases Excel
Microsoft releases the first version of Excel on 30 September 1985. The version is written for the Apple Macintosh. The Windows version of Excel is released in 1987. The spreadsheet program (the first with a graphical user interface) quickly outsells Lotus 1-2-3 and becomes Microsoft's flagship product. Today Excel is still often used as a testing tool.
1985 AutoTester released (Hayes)
Autotester, the frst commercial test tool for the PC (on MS-DOS) is released by Linda Hayes's company AutoTester. The company is based in Dallas, Texas.
1985 DOD-STD-2167 standard published
On 4 June 1985 the American Department of Defense publishes DOD-STD-2167; the standard that establishes 'the requirements to be applied during the development and acquisition of Mission-Critical Computer System software'. It includes formal testing.
1985 Reverse engineering (Rekoff)
In his paper On Reverse Engineering M. G. Rekoff introduces the term reverse engineering. Rekoff defines it as "the process of developing a set of specifications for a complex hardware system by an orderly examination of specimens of that system." Today the concept is also applied to software. It is a key part of software maintenance.
1986 V-model published (Rook)
In the article Controlling Software Projects, published in the IEEE Software Engineering Journal, Paul E. Rook introduces the V-model. Rook works for GEC Software Ltd. in London at that time. The model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing.
1986 SQE founded
David Gelperin and William Hetzel found Software Quality Engineering (SQE). The company is instrumental in setting the stage for the software industry to view testing as a distinct discipline.
1986 NCC publishes STARTS handbook
The British National Computer Centre publishes the the STARTS Purchasers' Handbook as a standard in order to "accelerate the uptake of best software engineering practice and methods".
1986 IEEE standard for V&V plans published
The IEEE Standard for Software Verification and Validation Plans (IEEE Std 1012-1986) is published.
1986 No Silver Bullet (Brooks)
In his paper "No Silver Bullet — Essence and Accidents of Software Engineering" Fred Brooks argues that "there is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity."
1987 Test, then code
Motto on the lapel pin of SQE as worn during the Fourth International Conference on Software Testing, Washington DC.
1987 Software reliability (Musa)
The seminal work Software Reliability: Measurement, Prediction, Application is published by John D. Musa, Anthony Iannino, and Kazuhira Okumoto. Software reliability has become a key part in software quality.
1987 Cost of Poor Quality (Harrington)
The Cost of Poor Quality (COPQ) is identified by H. James Harrington in his book Poor Quality Costs, published by the American Society for Quality Control (ASQC). The book documents IBM's approach to reporting the costs that result from poor quality.
1987 Use cases (Jacobson)
In his paper Object Oriented Development in an Industrial Environment, published in the OOPSLA ‘87 Proceedings, Ivar Jacobson introduces the concept of use cases and use case modeling from the customer requirements. Jacobson defines a use case as a special sequence of transactions, performed by a user and a system in a dialogue.
1987 Software design patterns (Beck, Cunningham)
In their presentation Using Pattern Languages for Object-Oriented Programs - to the OOPSLA-87 conference - Kent Beck and Ward Cunningham adapt the concept of the pattern language (Christopher Alexander) to object-oriented programming. The pattern language should guide the designer by providing a workable, reusable solution to solve recurring problems.
1988 Exploratory testing introduced (Kaner)
In the book Testing Computer Software Cem Kaner uses the term 'exploratory testing' for the first time.
1988 Testing Computer Software (Kaner)
The first edition of the book by Cem Kaner sets a new standard in software testing. The book is famous for its pragmatic and real-world oriented approach. The second edition of the book is co-authored by Jack Falk and Hung Q. Nguyen.
1988 CMM published (Humphrey)
Watts Humphrey - founder of the Software Engineering Institute's Software Process Program - publishes the Capability Maturity Model in the IEEE paper Characterizing the software process: a maturity framework
1988 The Growth of Software Testing (Gelperin, Hetzel)
In their ACM article The Growth of Software Testing David Gelperin and William Hetzel discuss four testing models and the evolution of testing.
1988 First defect tracking tool (DDTS)
Qualtrak (acquired by Pure Software in 1995) develops DDTS (Distributed default tracking system); a defect tracking system for the UNIX market.
1988 Spiral model (Boehm)
In his paper A Spiral Model of Software Development and Enhancement - published in the IEEE magazine Computer - Barry Boehm introduces the spiral model as a substitute for the waterfall model
1988 Segue Software founded
Segue Software is founded in Lexington, Massachusett by Laurence Kepple. It launches test automation tools such as SilkTest. The company is acquired by Borland in 2006.
1988 Fuzz testing introduced
The term 'fuzz' is coined by Barton Miller (Operating System Utility Program Reliability) to describe the use of random, unstructured data to investigate security flaws in a system.
1988 Gilb's risk principle
In the book Principles of Software Engineering Management authors Tom Gilb and Susannah Finzi introduce risk management in the principle that “If you don't actively attack the risks, they will actively attack you".
1989 SIGIST founded
The British Specialist Interest Group in Software Testing (SIGIST) is founded in 1989 by Geoff Quentin. Its first meeting is held at Imperial College in London. The meeting, during which four presentations (on risks, standards and reliability) are given, is attended by 29 people. The aim of the group is to share problems, successes and failures in testing, share techniques and share ideas of tools to support testing.
1989 Mercury Interactive founded
Mercury Interactive is founded in California by Amnon Landan and Arye Finegold. The company releases many test automation products until its acquisiton by HP in 2006.
1989 First version of LoadRunner
The performance testing tool LoadRunner is released by Mercury.
1989 ASQ Software Division founded
The Software Division of the American Society for Quality is founded. Taz Daughtrey is the first chairman of the division.
1990 Bug taxonomy (Beizer)
Boris Beizer provides his taxonomy (classification of bugs) in the second edition of the book Software Testing Techniques.
1990 Pesticide paradox (Beizer)
In the second edition of Software Testing Techniques Boris Beizer coins the term pesticide paradox to describe the phenomenon that the more you test software, the more immune it becomes to your tests.
1990 The Fifth Discipline (Senge)
In his book The Fifth Discipline Peter Senge discusses the five disciplines of the learning organization, focusing on group problem solving using the systems thinking method.
1990 The Machine That Changed the World (Womack)
In the book The Machine That Changed the World the authors James P. Womack, Daniel Roos, and Daniel T. Jones describe the lean manufacturing philiosophy. They base their work on a study of the automotive industry by MIT.
1991 ISO 9126 published
ISO/IEC 9126 Software engineering — Product quality is an international standard for the evaluation of software quality. Its quality model splits up quality into six characteristics.
1991 Start of the newsgroup comp.software.testing
The newsgroup is established to provide a forum for discussion about testing tools, techniques, and experiences among NASA, industry, and academia.
1991 First CAST report (Graham)
Unicom Seminars publishes the first CAST Report on Computer Aided Software Testing. The report is written by Dorothy Graham. It lists all tools available at that time, described in a standard way.
1991 Pure Software founded
Pure Software is founded by Reed Hastings, who later founded Netflix. Until its acquisition by Rational Software in 1997 it launches debugging and defect tracking software.
1991 RAD introduced (Martin)
In his book Rapid Application Development James Martin introduces rapid application development (RAD) as a development lifecycle designed to give much faster development and higher-quality results than those achieved with the traditional lifecycle. Martin extends on the methodology called Rapid Iterative Production Prototyping (RIPP) that was used at DuPont.
1991 Software Testing, Verification and Reliability (Wiley)
First issue of the journal Software Testing, Verification and Reliability is published by John Wiley & Sons. It is a quarterly international journal that publishes papers on theoretical and practical issues of software testing, verification and reliability.
1992 First STAR conference
The first STAR (Software Testing Analysis & Review) conference is held in Las Vegas. It is organised by Gelperin's and Hetzel's SQE.
1992 Rational Unified Process published
Ivar Jacobson, Grady Booch, and James Rumbaugh describe the generic process behind the Rational Unified Process (RUP) in the book The Unified Software Development Process. RUP is an iterative software development process framework created by Rational Software.
1992 First version of TTCN
The first version of the Testing and Test Control Notation (TTCN) - originally meaning Tree and Tabular Combined Notation - is published by the ETSI Centre for Testing and Interoperability. The language is launched as a specification of abstract test suites for conformance testing of International Telecommunications Union protocols. It is now promoted as a general purpose test language for distributed communicating systems.
1992 Orthogonal Defect Classification (Chillarege)
In the paper Orthogonal Defect Classification - A Concept for In-Process Measurements Ram Chillarege (IBM) publishes the concept of Orthogonal Defect Classification; the definition and capture of defect attributes that make mathematical analysis and modeling possible
1992 Technical debt (Cunningham)
Howard G. Cunningham introduces the debt metaphor in the experience report for the 92 OOPSLA conference. Cunnigham reasons that failure to consolidate code will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. His argument is that introducing unconsolidated code in a release (in order, for example, to release early) without refactoring the code after the release will result in debt that has to be repaid.
1993 First EuroSTAR (London)
The origin of Europe's premier software testing event. EuroSTAR is a sister conference of STAR. The conference is underwritten by SQE who runs it in conjunction with the British Computer Society. Dorothy Graham is programme chair for the first EuroSTAR.
1993 Scrum (Sutherland)
Jeff Sutherland, John Scumniotales, and Jeff McKenna develop software in scrum teams at Easel Corporation and for the first time call it Scrum.
1993 Decision tableau method (Mosley)
Daniel J. Mosley illustrates his Decision Tableau method in The Handbook of MIS Application Software Testing.
1993 W-model introduced (Herzlich)
In his presentation The Politics of Testing Paul Herzlich introduces the W-model. The model attempts to address shortcomings in the V-Model. Herzlich holds the presentation during the first EuroSTAR conference in London.
1993 SUMI questionnaire published
First publication of the SUMI (Software Usability Measurement Inventory) questionnaire at the University College Cork.
1993 Classification Tree Method (Grimm)
In their paper Classification Trees for Partition Testing Matthias Grochtmann and Klaus Grimm publish the Classification Tree Method.
1994 Usability Inspection Methods (Nielsen a.o.)
The book Usability Inspection Methods, edited by Jakob Nielsen and Robert L. Mack, is the first comprehensive work on usability. It is a practical guide containing strategies, tools, and techniques for the evaluation of usability.
1994 MoSCoW principle introduced (Clegg)
Dai Clegg (Oracle UK Consulting) publishes this popular prioritisation technique in the book CASE Method Fast-Track: A RAD Approach.
1994 Goal Question Metric approach (Basili)
Victor R. Basili, Gianluigi Caldiera and H. Dieter Rombach publish the Goal Question Metric (GQM) approach in their paper The Goal Question Metric Approach. The aim is to provide a top-down definition of measurements for feedback and evaluation, starting with the goals of the organization.
1994 A Metrics Suite for Object Oriented Design
Shyam R Chidamber and Chris F Kemerer publish their paper A Metrics Suite for Object Oriented Design. It contains a new set of software metrics for object oriented design with metrics such as Weighted Methods Per Class and Depth of Inheritance Tree.
1994 First Chaos report (Standish Group)
The Standish Group starts the Chaos report, a continuing study to identify the scope of software project successes and failures, the major factors that cause software projects to fail, and the key ingredients that can reduce software project failures.
1995 TMap published (Pol a.o.)
Martin Pol, Ruud Teunissen and Erik van Veenendaal publish their test management approach for structured testing. The first publication of the book is in Dutch.
1995 DSDM published
The DSDM Consortium publishes the Dynamic Systems Development Method; an iterative and incremental approach to software development
1995 First release of Winrunner
The record & playback tool Winrunner is released by Mercury.
1995 SAST formed
The Swedish Association for Software Testing (SAST) is formed by Anna Eriksson and Sigrid Eldh as a platform to exchange experiences and knowledge related to testing.
1995 Anti-pattern (Koenig)
The term 'anti-pattern' is coined by Andrew Koenig in his paper Patterns and Antipatterns published in the Journal of Object-Oriented Programming. An anti-pattern is a design pattern that appears obvious but is ineffective or far from optimal in practice. A pattern in software design is a general reusable solution or a best practice.
1996 Google online
Larry Page and Sergey Brin launch Google; the internet service provider that rapidly becomes the most visited website in the world.
1996 First Extreme Programming project
The first Extreme Programming project (Chrysler Comprehensive Compensation System) is started March 6, 1996 by Kent Beck, Ron Jeffries and Howard G. Cunningham.
1996 TMM developed
The Testing Maturity Model is developed at the Illinois Institute of Technology.
1996 First SQC conference
The first Software & Systems Quality conference (SQC) is organized by the German company SQS. The conference is now known as ignite and is held in many countries.
1996 Cornering the Chimera (Dromey)
In his paper Cornering the Chimera R. Geoff Dromey proposes a quality model that solves the intangeability of the ISO 9126 Quality characteristics. Dromey links components properties to higher level quality characteristics.
1996 Heuristic Test Strategy Model (Bach)
James Bach introduces guideword heuristics to software testing in his exploratory testing class. For this purpose Bach uses the Tripos Model which is expanded later and renamed Heuristic Test Strategy Model. During the class Bach also introduces the famous mnemonic and heuristic SFDPO.
1996 Pairwise testing (Cohen)
In their paper The Combinatorial Design Approach to Automatic Test Generation, authors David Cohen, Siddhartha Dalal, Jesse Parelius and Gardner Patton introduce pairwise testing. They base their combinatorial design theory to generate tests on the usage of the Automatic Efficient Test Generator (AETG), during projects at Bellcore.
1997 TestNet founded
The Dutch TestNet society is founded on the 9th of June 1997 as a platform for the exchange of knowledge about software testing. The initiative for the foundation of TestNet is taken by Martin Pol, Ingrid Ottevanger, Erik van Veenendaal and Jos Trienekens during the first EuroSTAR conference in the Netherlands (Amsterdam, 1996).
1997 First version of UML
UML (Unified Modeling Language) is created by James Rumbaugh, Grady Booch and Ivar Jacobson at Rational Software. The UML 1.1 specification is adopted by the Object Management Group in 1997. The language becomes the basis for model-based testing.
1997 First Los Altos Workshop on Software Testing
The first Los Altos Workshop is held, hosted by Cem Kaner and Brian Lawrence. It is a periodic gathering of senior test practitioners.
1997 The Cathedral and the Bazaar (Raymond)
In his essay The Cathedral and the Bazaar author Eric S. Raymond discussed the software development method that was used by Linus Torvalds in creating Linux. Raymond describes it as 'release early and often, delegate everything you can, be open to the point of promiscuity'. He mentions 19 guidelines for creating good open source software. Raymond also introduces Linus's Law which states that "given enough eyeballs, all bugs are shallow". It means that "Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone."
1997 Death march projects (Yourdon)
In his book Death March: The Complete Software Developer's Guide to Surviving "Mission Impossible" Projects Edward Yourdon investigates extensively why 'death march' projects exist and offer some guidance to surviving this type of project. The death march project is a common situation in software development in which work is continued on a project that is doomed to fail.
1998 ISEB Software Testing Certification
The first European certification for software testers is created by the British Information Systems Examinations Board. The first course is taught by Dorothy Graham and Mark Fewster.
1998 BCS standard for Software component testing
The Standard for Software component testing (BS 7925-2) is published by the Testing Standards Working Party (sponsored by the BCS SIGiST).
1998 Apache jMeter released
Version 1.0 of Apache jMeter is published on December 15 1998. The tool is authored by Stefano Mazzocchi. JMeter is an open source application designed to load test functional behavior and measure performance.
1998 Bugzilla released
The first version of the popular defect tracking tool Bugzilla is release. Bugzilla was originally developed by Terry Weissman in 1998 for the nascent Mozilla.org project, as an open source application.
1998 Personas (Cooper)
In his book The Inmates Are Running the Asylum Alan Cooper introduces the use of personas as a practical interaction design tool. He recommends that software be designed for single archetypal users.
1999 Context-Driven Testing (Bach a.o.)
Cem Kaner, James Bach, Brian Marick and Bret Pettichord start the Context-Driven school of testing based on the principle that 'The value of any practice depends on its context'.
1999 Software Test Automation (Fewster & Graham)
The book by Mark Fewster and Dorothy Graham is a standard work in the field of automation. It comprehensively treats software test automation issues, strategies and tactics.
1999 Test Process Improvement (Koomen, Pol)
Tim Koomen and Martin Pol publish their approach to assessing and improving test processes.
1999 First version of QuickTest
Early 1999 Mercury interactive releases QuickTest as an easier approach to creating and maintaining SAP automated tests. It later becomes QuickTest Professional.
1999 Better Software magazine launched
Launched in January 1999 as Software Testing and Quality Engineering, Better Software magazine is a magazine 'for software professionals who care about quality'. The magazine is published by SQE.
1999 CSTP certification
The Certified Software Test Professional (CSTP) certification by International Institute for Software Testing.
1999 Software Metrics: Ten Traps To Avoid (Wiegers
In the paper Software Metrics: Ten Traps To Avoid Karl E. Wiegers examines 'ten traps that can sabotage the unsuspecting metrics practitioner'.
1999 TestFrame published
The test methodology TestFrame is published by Hans Buwalda,Dennis Jansen and Iris Pinkster, working for the British/Dutch IT company LogicaCMG. Buwalda worked on the method since 1994 when it originated as CMG:CAST. TestFrame focuses on the use of test automation.
2000 Session-based testing (Bach)
Jonathan and James Bach publish the session-based testing method that combines accountability with exploratory testing.
2000 Professional Tester magazine launched
First publication of Professional Tester magazine. It is published by the London based publishing house Test Publishing Limited. The magazine was 'relaunched' in 2010.
2000 Continuous integration (Fowler)
In his article Continuous Integration Martin Fowler discusses a software development practice where members of a team integrate their work frequently leading to multiple integrations per day.
2001 Agile Manifesto is published
A conference in Utah by 17 representatives from different development methodologies leads to the Agile Manifesto in which the Twelve Principles of Agile Software are published.
2001 Lessons Learned in Software Testing
James Bach, Cem Kaner and Bret Pettichord publish their experiences with context-driven testing in Lessons Learned in Software Testing.
2001 StickyMinds.com launched
The popular community website for software testing and software quality is launched by SQE.
2001 SaaS model introduced
The acronym SaaS for Software as a Service is first used in the article Software as a Service: Strategic Backgrounder. The article is published by Software & Information Industry's (SIIA) eBusiness Division. The Software as a Service model defines the distribution of application through the internet.
2001 First PSQT conference
The first conference on Practical Software Quality and Testing (PSQT) is held in Orlando, Florida. It is organized by Software Dimensions and the International Institute for Software Testing.
2001 OpenSTA released
The first version of the open source web load and performance testing tool OpenSTA is released. It is originally released by the French Cyrano SA group.
2001 First Rapid Testing class
James Bach expands his exploratory software testing class into the Rapid Software Testing (RST) class based on the ideas of the context-driven school of software testing. Michael Bolton expands the Rapid Software Testing class adding his own material and starts teaching RST in 2003.
2002 ISTQB founded
The International Software Testing Qualifications Board is founded in Edinburgh in November 2002.
2002 Test-Driven Development (Beck)
In the book Test Driven Development: By Example Kent Beck publishes the software development technique in which a test is written before the functionality is coded.
2002 IBM Rational Functional Tester
Originally released as RobotJ, IBM Rational Functional Tester is IBM's key automated test script generator.
2002 Software Attacks (Whittaker)
In his book How to Break Software: A Practical Guide to Testing James Whittaker publishes a total of 23 attacks to expose software errors. The attacks are designed for the user interface and for the system.
2002 JIRA 1.0 released
The first version of the popular bug tracking tool JIRA is released by the Australian software company Atlassian Software.
2002 Framework for Integrated Test (Cunningham)
Framework for Integrated Test (Fit), built by Ward Cunningham, is an open-source tool for automated customer tests. The tool integrates test development into specification and test execution into programming using fixtures. It gives customers and programmers a way to communicate precisely about their software.
2003 Schools of Software Testing (Pettichord)
Bret Pettichord publishes the paper in which he classifies software testing ideas into five schools.
2003 SPICE (ISO 15504) published
The first part (part 2) of ISO/IEC 15504 is published in October 2003 (the working draft was started in 1993). ISO/IEC 15504 is also known as SPICE (Software Process Improvement and Capability Determination). It is a framework for process assessment.
2003 Domain-Driven Design (Evans)
In his book Domain-Driven Design: Tackling Complexity in the Heart of Software Eric Evans offers a set of design best practices, experience-based techniques, and fundamental principles that facilitate the development of software projects facing complex domains.
2003 Association for Software Testing founded
The Association for Software Testing is founded in 2003 and incorporated as a professional non-profit association in 2004. Its initial board of directors consists of Patrick J. Schroeder, James Patrick McGee, Cem Kaner, Andrew David Tinkham, Douglas Hoffman, Tim van Tongeren and Hung Quoc Nguyen. In 2006 the association organizes the first CAST conference. AST is focused on supporting the development of professionalism in software testing,
2003 Agile Testing Quadrant (Marick)
On his weblog Brian Marick publishes the Agile Testing Quadrant in August 2003. The quadrant combines two distinctions in testing; the distiction between business facing or technology facing testing and the distinction between support programming or critique the product.
2004 Selenium developed
The popular web application test tool Selenium is developed at ThoughtWorks (Chicago) by Jason Huggins.
2004 STP magazine launched
Software Test & Performance magazine is launched by BZ Media. It is now called Software Test & Quality Assurance magazine.
2005 TMMi published
The Test Maturity Model Integration is published by the TMMi Foundation. The model serves as a standard for assessing and improving the testing process.
2006 First CAST conference
The first Conference of the Association for Software Testing is held in june 2006 in Indianapolis, Indiana. The keynote speakers are James Bach and Cem Kaner.
2007 ISO forms working group for ISO 29119
In May 2007 ISO forms a working group to develop a new standard on software testing (ISO 29119). The standard will replace a number of existing IEEE and BSI standards for software testing. Dr. Stuart Reid is chairman of the group.
2007 Software Testing Club founded
The Software Testing Club is founded in 2007 by Rosie Sherry after a clear need for a community for software testers with a quality approach.
2008 Testing Experience magazine launched
The magazine Testing Experience is launched by the German company Díaz & Hilterscheid.
2008 Software Testing as a Service (STaaS)
The term Software Testing as a Service (STaaS) is coined by Leo van der Aalst in a presentation to the Pacific Northwest Software Quality Conference. Software Testing as a Service (STaaS) is a model of software testing used to test an application as a service provided to customers across the Internet.
2009 T.E.S.T magazine launched
In March 2009 the first issue of the British online magazine T.E.S.T is published by 31Media.
2009 Testing vs. Checking (Bolton)
In his weblog article Testing vs. Checking Michael Bolton distinguishes testing from checking. Checking, according to Bolton, is confirmation, verification and validation while testing is the process of exploration, discovery, investigation, and learning.
2009 Weekend Testers founded
Weekend Testers, formely known as Bangalore Weekend Testers is founded by Parimala Shankariah, Sharath Byregowda, Manoj Nair and Ajay Balamurugadas in August 2009. Its mission is to be "A platform for software testers to collaborate, test various kinds of software, foster hope, gain peer recognition, and be of value to the community." Weekend Testers has chapters in India, Europe, Australia and the United States.