In the talk, we focus on algorithms for solving well-structured large-scale convex programs in the case where huge problem’s sizes prevent processing it by polynomial time algorithms and thus make computationally cheap first order optimization methods the methods of choice. We overview significant recent progress in utilizing problem's structure within the first order framework, with emphasis on algorithms with dimension-independent (and optimal in the large-scale case) iteration complexity being the target accuracy. We then discuss the possibility to further accelerate the first order algorithms by randomization, specifically, by passing from expensive in the extremely large scale case precise deterministic first order oracles to their computationally cheap stochastic counterparts. Applications to be discussed include SVM's, minimization, testing sensing matrices for "goodness" in the Compressed Sensing context, low-dimensional approximation of high-dimensional samples, and some others.