Manipulating data files

Introduction

One of the most common tasks for scientists and engineers is working with data sets that have been generated by some external process or collector. If they are lucky, the data are stored in a file that has a standard format and can then be read into other programs such as Mathematica with ease using that program's importing functionality. Oftentimes, however, data are stored in files with nonstandard formats and reading that file into a program such as Mathematica requires some manual processing of that file to extract the required parts.

In this section we will walk through the steps of reading, manipulating, and visualizing a dataset that consists of solar radiation data collected by the Renewable Resource Data Center, an organization that is managed by the US Department of Energy (interested readers should visit rredc.nrel.gov).