|
Today I read a comic strip that had to do with testing in IT. In the strip the main character is upset that the test cases they just completed, didn’t have any relevance to the real world. He was lamenting the fact that he had just spent two weeks testing something that was meaningless. The manager in the comic strip said that they would use the test results anyway because it is the only data they have. At that, the main character says something unkind and leaves. The boss asks why he is so cranky and another character says it has something to do with data. Unfortunately, this comic strip reflects reality. There are times when the testing being done for a project, doesn’t bear any relevance to the real world. The purpose of testing is to determine if the code changes will accomplish the business requirements of the project. Test cases should be written to test every step in the process and every piece of code, to ensure that it all operates correctly. Many test cases test the mechanics of the process. However, there can be disastrous results if testing does not include all of the system that uses the data in the project. Case in point… One time a data source changed the way data was coming in an address field. The job that processed the address records failed because the field started coming in left justified instead of right justified. The issue needed to be fixed immediately, so a solution was designed to handle left and right justified data. After testing those two situations in the development environment, test cases were written. The test cases written tested the data before it was converted, and then after it was converted. The processes were run and the data was verified. So, the solution was promoted to the production environment on Friday. The following Monday, an issue was raised because over 70,000 change address letters were mailed to clients. The worse part was, most of the addresses hadn’t changed, but the data had changed due to the new code that was promoted. It turns out that the test cases, verified the size of the files by comparing bytes before and bytes after, however, the data wasn’t examined after the processing. The test cases tested the mechanics of the solution, but didn’t test the downstream processes to ensure that the solution wouldn’t cause more harm. It took over two weeks to backout the errors caused by this promotion. If the test cases had involved the downstream processing, and data validation by eyes on, rather than just comparing file sizes, this extra work would have been averted. Just like the main character in the comic strip, many people were cranky over something so little as data.
 |
Subscribe to this comment's feed