Granted, the population has increased somewhat, but the data don't appear to support the claim that there were vast hordes of clerical workers who were "hurt" by computers.
The problem appears to be the same as the "paperless office" and the "productivity paradox". As computers became more powerful and common, rather than reducing the demand for secretaries or paper, or for making business systems more efficient, computers increased the amount of information (including on paper) being processed by more than enough to offset the expected changes. (I don't have a reference, because this is my own speculation based on a great deal of reading on how computers were expected to be, and actually have been, used.)
The numbers don't back that up. There were 13 million clerical workers in the United States in 1970 (a 40% increase from Mad Men's 1960, btw):
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_...
There are now over 21 million: http://www.bls.gov/oes/current/oes430000.htm
Granted, the population has increased somewhat, but the data don't appear to support the claim that there were vast hordes of clerical workers who were "hurt" by computers.