I am working as a scientist at EPFL
with Prof. Martin Jaggi in the Machine Learning and Optimization Laboratory (MLO).
with Prof. Martin Jaggi in the Machine Learning and Optimization Laboratory (MLO).
Research Interests
- complexity analysis of (randomized) optimization algorithms, in serial, parallel and distributed settings
- optimization algorithms for high-dimensional and/or structured problems
Workshop
-
Advances in ML: Theory meets practice,
co-organized with Dan Alistarh
28. January 2018, Lausanne, Switzerland. This workshop is a part of the Applied Machine Learning Days 2018.
Full Program and Abstracts
Reviewing for
- Conferences: Symposium on Discrete Algorithms (SODA)19Genetic and Evolutionary Computation Conference (GECCO)1817161514International Conference on Machine Learning (ICML)1817Neural Information Processing Systems (NIPS)18Parallel Problem Solving from Nature (PPSN)18Symposium on the Theory of Computing (STOC)18
- Journals: European Journal of Operational Research (EJOR)Journal of Machine Learning Research (JMLR)Markov Processes And Related Fields (MPRF)Mathematical Programming (MAPR)Numerical Algebra, Control and Optimization (NACO)Optimization Methods and Software (OMS)SIAM Journal on Optimization (SIOPT)
Education and Work
- From Nov 1 2014 to Okt, 31, 2016, I worked with Prof. Yurii Nesterov and Prof. François Glineur at the Center for Operations Research and Econometrics (CORE). I was also member of the Institute of Information and Communication Technologies, Electronics and Applied Mathematics (ICTEAM).
- From Sep 15, 2010 to Sep 30, 2014, I was a PHD student in Prof. Emo Welzl's research group, supervised by Prof. Bernd Gärtner and Christian Lorenz Müller.
- Until Jun 31, 2012, I was also member of Prof. Ivo Sbalzarini's MOSAIC group. The group now moved to the Max Planck Institute of Molecular Cell Biology and Genetics (MPI-CBG) in Dresden.
- From Sep 2005 to Mar 2010 I did my Bachelor and Master in Mathematics at ETH Zurich.
Publications
Refereed Publications
- On Matching Pursuit and Coordinate DescentBibIn:ICMLPMLR 802018
- Adaptive balancing of gradient and update computation times using global geometry and approximate subproblemsSlidesBibIn:AISTATSPMLR 84, 1204–12132018
- Safe Adaptive Importance SamplingVideoSlidesPosterBibSpotlight talk, In:NIPS30, 4381–43912017
- On the existence of ordinary trianglesBibComputational Geometry66, 28–312017
- Approximate Steepest Coordinate DescentPosterBibIn:ICMLPMLR 70, 3251–32592017
- Efficiency of accelerated coordinate descent method on structured optimization problemsBibSIAM Journal on Optimization27:1, 110–1232017
- Variable Metric Random PursuitBibMathematical Programming156(1), 549–5792016
- On two continuum armed bandit problems in high dimensionsBibTheory of Computing Systems58:1, 191–2222016
- On low complexity Acceleration Techniques for Randomized OptimizationSupplBibIn:PPSN XIIISpringer, 130–1402014
- Optimization of Convex Functions with Random PursuitBibSIAM Journal on Optimization23:2, 1284–13092013
- On spectral invariance of Randomized Hessian and Covariance Matrix Adaptation schemesPosterBibIn:PPSN XIISpringer, 448–4572012
- On Two Problems Regarding the Hamiltonian Cycle GameBibThe Electronic Journal of Combinatorics16(1)2009
Preprints / Various:
- Don't Use Large Mini-Batches, Use Local SGDBibTechnical Report2018
- Global linear convergence of Newton's method without strong-convexity or Lipschitz gradientsBibTechnical Report2018
- Local SGD Converges Fast and Communicates LittleBibTechnical Report2018
- SVRG meets SAGA: k-SVRG — A Tale of Limited MemoryBibTechnical Report2018
- Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order OptimizationAbstractBibTechnical Report2018We present the first accelerated randomized algorithm for solving linear systems in Euclidean spaces. One essential problem of this type is the matrix inversion problem. In particular, our algorithm can be specialized to invert positive definite matrices in such a way that all iterates (approximate solutions) generated by the algorithm are positive definite matrices themselves. This opens the way for many applications in the field of optimization and machine learning. As an application of our general theory, we develop the first accelerated (deterministic and stochastic) quasi-Newton updates. Our updates lead to provably more aggressive approximations of the inverse Hessian, and lead to speed-ups over classical non-accelerated rules in numerical experiments. Experiments with empirical risk minimization show that our rules can accelerate training of machine learning models.
- Stochastic continuum armed bandit problem of few linear parameters in high dimensionsBibTechnical Report2013
- Random Pursuit in Hilbert SpaceBibTechnical ReportCGL-TR-882013
- Matrix-valued Iterative Random ProjectionsBibTechnical ReportCGL-TR-872013
Theses
- Convex Optimization with Random PursuitBibPhD thesis in Theoretical Computer ScienceETH Zurich2014
- Graph sparsification and applicationsBibMaster thesis in MathematicsETH Zurich2010
- On two problems regarding the Hamilton Cycle GamePaperBibBachelor thesis in MathematicsETH Zurich2008
Talks / Posters / Workshops
- Communication Efficient Variants of SGD for Distributed Computing(EULER Seminar)2–14 September2018research visit François Glineur, Yurii Nesterov, UCLouvainLouvain-la-NeuveBelgium
- Communication Efficient SGD through Quantization with Memory30 August2018MittagsseminarETH ZürichSwitzerland
- On Matching Pursuit and Coordinate Descent(talk by Francesco)10–15 July201835th International Conference on Machine Learning (ICML)StockholmSweden
- Approximate Composite Minimization: Convergence Rates and Examples2–6 July201823rd International Symposium on Mathematical Programming (ISMP)BordeauxFrance
- Adaptive balancing of gradient and update computation times(talk by Praneeth)9–11 April2018The 21st International Conference on Artificial Intelligence and Statistics (AISTATS)Playa Blanca, LanzaroteSpain
- 19–20 February2018Swiss Joint Research Center Workshop 2018EPFL LausanneSwitzerland
- 27–30 January2018Applied Machine Learning DaysEPFL LausanneSwitzerlandWorkshop
- Adaptive importance sampling for convex optimization(invited)15–18 January2018Optimization for Learning WorkshopPontificia Universidad Católica de Chile, SantiagoChile
- Adaptive importance sampling for convex optimization2–13 January2018research visit Hemant Tyagi, Alan Touring InstituteLondonUK
- Safe Adaptive Importance Sampling4–9 December2017Thirty-first Annual Conference on Neural Information Processing Systems (NIPS)Long BeachUSAPoster
- 27 November – 1 December2017Optimization, Statistics and Uncertainty Simons Institute for the Theory of Computing, BerkelyUSA
- Approximate Steepest Coordinate Descent3–21 October2017research visit Peter Richtárik, KAUSTThuwalKSA
- Approximate Steepest Coordinate Descent6–11 August201734th International Conference on Machine Learning (ICML)SidneyAustraliaPoster
- 14–16 June2017The Summer Research Institute 2017EPFL LausanneSwitzerland
- 12–14 June2017Google Machine Learning SummitGoogle ZürichSwitzerland
- Efficiency of Coordinate Descent on Structured Problems22–25 May2017SIAM Conference on Optimization (OP17)VancouverCanada
- 2–3 February2017Swiss Joint Research Center Workshop 2017Microsoft CambridgeUK
- 30–31 January2017Applied Machine Learning DaysEPFL LausanneSwitzerland
- 5–10 December2016Thirtieth Annual Conference on Neural Information Processing Systems (NIPS)BarcelonaSpain
- Randomized Algorithms for Convex Optimization18 October2016Operations Research SeminarCORE, Louvain-la-NeuveBelgium
- Efficiency of Random Search on Structured Problems6–11 August20165th International Conference on Continuous Optimization (ICCOPT)TokyoJapan
- 6–10 June201614th Gremo Workshop on Open ProblemsSt. NiklausenSwitzerland
- Adaptive Quasi-Newton Methods6 June20169th Combinatorial Algorithms DayETH ZürichSwitzerland
- 7–12 February2016Optimization Without Borders—dedicated to the 60th birthday of Yuri NesterovLes HouchesFrance
- Efficient Methods for a Class of Truss Topology Design Problems28–29 January201630th annual conference of the Belgian Operational Research Society (ORBEL)Louvain-la-NeuveBelgium
- 23 November2015IAP DYSCO Study Day: Dynamical systems, control and optimizationLeuvenBelgium
- 26 October – 20 November2015SOCN Grad Course: Randomized algorithms for big data optimizationLouvain-la-NeuveBelgium
- Accelerated Random Search8–10 July201513th EUROPT Workshop on Advances in Continuous OptimizationEdinburghUK
- 1–5 June201513th Gremo Workshop on Open ProblemsFeldisSwitzerland
- Solving generalized Laplacian linear systems1 June20158th Combinatorial Algorithms DayETH ZürichSwitzerland
- 6–8 May2015Optimization and Big Data 2015, Workshop, Trek and ColloquiumEdinburghUK
- 12 November2014IAP DYSCO Study Day: Dynamical systems, control and optimizationGentBelgium
- Optimization of convex function with Random Pursuit4 November2014Simons FoundationNew YorkUSA
- On Low Complexity Acceleration Techniques for Randomized Optimization13–17 September201413th International Conference on Parallel Problem Solving From Nature (PPSN)LjubljanaSlovenia
- 30 June – 4 July201412th Gremo Workshop on Open ProblemsVal SinestraSwitzerland
- 30 June20147th Combinatorial Algorithms DayETH ZürichSwitzerland
- Probabilistic Estimate Sequences4–5 April2014Workshop on Theory of Randomized Search Heuristics (ThRaSH) 2014JenaGermany
- Probabilistic Estimate Sequences25 March2014TAO reserach seminarINRIA Saclay, Île-de-FranceFrance
- Natural Gradient in Evolution Strategies17 October2013MittagsseminarETH ZürichSwitzerland
- Optimization and Learning with Random Pursuit30 September – 2 Oktober2013CG Learning Review MeetingAthensGreece
- 30 June – 5 July2013Dagstuhl Seminar "Theory of Evolutionary Algorithms"WadernGermany
- 24–28 June201311th Gremo Workshop on Open ProblemsAlp SellamattSwitzerland
- 24 June20136th Combinatorial Algorithms DayETH ZürichSwitzerland
- Stochastic Bandits30 May2013MittagsseminarETH ZürichSwitzerland
- 6 April – 8 May2013Research visit with Christian Müller and Jonathan GoodmanCourant Institute of Mathematical Sciences, New York UniversityUSA
- Variable Metric Random Pursuit13–14 December2012CG Learning Review MeetingBerlinGermany
- Variable Metric Random Pursuit4 December2012MittagsseminarETH ZürichSwitzerland
- On spectral invariance of Randomized Hessian and Covariance Matrix Adaptation schemesPoster1–5 September201212th International Conference on Parallel Problem Solving From Nature (PPSN)TaorminaItaly
- Convergence of Local Search19–24 August201221st International Symposium on Mathematical Programming (ISMP)TU BerlinGermany
- 20–22 June201212th International Workshop on High Performance Optimization (HPOPT): Algorithmic convexity and applicationsDelft University of TechnologyThe Netherlands
- 4–8 June201210th Gremo Workshop on Open ProblemsBergünSwitzerland
- 4 June20125th Combinatorial Algorithms DayETH ZürichSwitzerland
- Convergence of Local Search2–3 May2012Workshop on Theory of Randomized Search Heuristics (ThRaSH) 2012Lille/Villeneuve d'AscqFrance
- The Heavy Ball Method3 April2012MittagsseminarETH ZürichSwitzerland
- Advertising Randomized derivative-free optimization11–14 March2012The First ETH-Japan Workshop on Science and ComputingEngelbergSwitzerland
- 7–9 March2012The First ETH-Japan Symposium for the Promotion of Academic ExchangesETH ZürichSwitzerland
- Gradient-free optimization with Random Pursuit15 December2011CG Learning Review MeetingZürichSwitzerland
- Dimension reduction with the Johnson-Lindenstrauss Lemma29 September2011MittagsseminarETH ZürichSwitzerland
- 30 August – 2 September2011International Conference on Operations ResearchZürichSwitzerland
- Randomized Derivative-Free Optimization, a survery of different methodsPoster8–11 August2011MADALGO & CTIC Summer School on High-dimensional Geometric ComputingAarhus UniversityDanmark
- Random derivative-free optimization of convex functions using a line search oracle8–9 July2011Workshop on Theory of Randomized Search Heuristics (ThRaSH) 2011CopenhagenDanmark
- 4–8 July201138th International Colloquium on Automata, Languages and Programming (ICALP)ZürichSwitzerland
- 9–11 June2011CG Learning Kick-off WorkshopParisFrance
- 28–30 March201127th European Workshop on Computational Geometry (EuroCG)MorschachSwitzerland
- 7–10 February20119th Gremo Workshop on Open ProblemsWergensteinSwitzerland
- Principles of self-adaptation in randomized optimization16 December2010MittagsseminarETH ZürichSwitzerland
- Graph sparsification with applications11 March2010Institute for Operations Research (IFOR)ETH ZürichSwitzerland
Student Projects
- Alberto ChiappaReal-time recognition of industry-specific context from mobile phone sensor dataMaster Thesis (industry project with Schindler)2018
- Jean-Baptiste CordonnierDistributed Machine Learning with Sparsified Stochastic Gradient DescentMaster Thesis2018
- Arthur DeschampsAsynchronous Methods in Machine LearningBachelor Project2018
- Chia-An YuCompression for Stochastic Gradient DescentMaster Project2018
- Frederik KünstnerFully Quantized Distributed Gradient DescentMaster Project2017
- Pooja KulkarniSteepest Descent and Adaptive SamplingSummer Internship2017
- Anastasia KoloskovaSteepest CD and LSHSummer Internship2017
- William Borgeaud DitAdaptive Sampling in Stochastic Coordinate DescentMaster Project2017
- Joel CastellonComplexity analysis for AdaRBFGS: a primitive for methods between first and second orderSemester Project2017
- Alberto ChiappaAsynchronous updates for stochastic gradient descentMaster Project2017
- Marina MannariFaster Coordinate Descent for Machine Learning through Limited Precision OperationsMaster Thesis2017
- Anant RajSteepest Coordinate DescentInternship2017
Teaching
Teaching Assistance
- Game TheorySpring 2016
- Algorithms LabFall 2013
- Informaticsfor mathematics and physics (in C++) (head assistant)Fall 2013
- Algorithms LabFall 2012
- Informaticsfor mathematics and physics (in C++)Fall 2012
- Approximation Algorithms and Semidefinite Programming(head assistant)Spring 2012
- Algorithms LabFall 2011
- Informaticsfor mathematics and physics (in C++)Fall 2010
- Analysis Ifor mechanical engineeringFall 2009
- Analysis IIfor mechanical engineeringSpring 2009
- Analysis Ifor mechanical engineeringFall 2008
- Analysis IIfor mechanical engineeringSpring 2008
- Complex AnalysisFall 2007