Networks are everywhere and we are confronted with many networks in our daily life. Networks such as Internet, World Wide Web, social, biological and economical networks have been subject to extensive studies in the last decade. The volume of publications and investments are sharply increasing and the applications are spreading widely over various disciplines. One of the most important problems in this context, which is the subject of interest of this thesis, is how the information is processed in a dynamical network and how a dynamical network can be exploited to solve information processing tasks. To study the problem more systematically, the information processing dynamical networks (IPDNs) are divided into two sub categories namely, (i) autonomous IPDNs in which the input information is static and is given as the initial state of the network and (ii) non-autonomous IPDNs in which the input information is injected by an input signal. In this thesis, synchronizability of dynamical networks, community detection on dynamical networks and skill acquisition in reinforcement learning (RL) using dynamical networks have been considered as the problems for the category of autonomous IPDNs. Furthermore, adaptation of the internal weights of a special type of recurrent neural networks, so called reservoir-based RNN (RRNN), and multi tasking in RRNNs have been investigated in the non-autonomous IPDNs category. First, we discuss the interplay between the structure of a complex network and its dynamical behavior and in particular the synchronizability of the network. There are several interpretations of synchronizability in the literature which are categorized in four main groups that do not coincide in general. One of these interpretations uses the second smallest eigenvalue of the graph Laplacian, the so called algebraic connectivity, which is an important measure for various applications in dynamical networks. We introduce a new lower bound for the algebraic connectivity and prove that it is always larger or equal to the previously published lower bounds. In addition, the proposed bound helps to explain the most effective parameters for the synchronizability of the network from the algebraic connectivity point of view. Having found the most influential parameters, we propose two algorithms for enhancing synchronizability of a given network by efficient rewiring of the links and by assigning proper weights to the links, respectively. The rewiring algorithm converts an arbitrary graph, irrespective of the topology, to a class of networks, so called super democratic networks that are identified by their homogenous degree and load distribution and their very low average path length. On the other hand, the weighting algorithm utilizes a novel graph theoretic measure, the so called neighboring graph to determine the proper weights of the links. The optimality of the weighting algorithm is proved and it is shown by numerical simulation on the benchmark networks