Elements of a PyTorch Deep Learning Model (1)- Tensors, Autograd and Optimization
Explore the key elements of PyTorch deep learning models, including tensors, autograd for gradient computation, and optimization techniques like SGD and Adagrad.
TENSOR AND COMMON TENSOR OPERATIONS
In PyTorch, a tensor is a multi-dimensional array that can represent scalars, vectors, matrices, or higher-dimensional data. Tensor refers to the generalization of vectors and matrices to an arbitrary number of dimensions. torch module provides extensive library of operations on them.
Each tensor in PyTorch has three attributes that uniquely identify it:
1. Data: The data attribute refers to the actual numerical values stored in the tensor. It represents the content of the tensor and defines its shape. The data can be stored in various data types, such as floats (`torch.float32`), integers (`torch.int64`), or booleans (`torch.bool`), depending on the requirements of the application.
2. Shape: The shape attribute describes the dimensions of the tensor. It is a tuple of integers that specifies the size of each dimension. For example, a 2D tensor with shape `(3, 4)` represents a matrix with 3 rows and 4 columns. The shape attribute is important for understanding the structure and layout of the data stored in the tensor.
3. Device: The device attribute indicates the device (e.g., CPU or GPU) where the tensor is located. PyTorch supports tensor computations on both CPUs and GPUs. By default, tensors are created on the CPU, but we can explicitly move them to a GPU for accelerated computation using the `.to()` method. The device attribute is crucial for managing the placement and movement of tensors across devices.
PyTorch Tensors are very close to NumPy arrays . In fact, PyTorch features seamless interoperability with NumPy. Compared with NumPy arrays, PyTorch tensors have added advantage that both tensors and related operations can run on the CPU or GPU.
The second important thing that PyTorch provides allows tensors to keep track of the operations performed on them that helps to compute…
