Высокопроизводительные вычислительные платформы: текущий статус и тенденции развития
Авторы
-
А.С. Антонов
-
И.В. Афанасьев
-
Вл.В. Воеводин
Ключевые слова:
суперкомпьютер
производительность
эффективность
процессор
Top500
параллелизм
ускоритель
коммуникационная сеть
Аннотация
В данной статье представлен обзор современного состояния суперкомпьютерной техники. Обзор сделан с разных точек зрения — начиная от особенностей построения современных вычислительных устройств до особенностей архитектуры больших суперкомпьютерных комплексов. В данный обзор вошли описания самых мощных суперкомпьютеров мира и России по состоянию на начало 2021 г., а также некоторых менее мощных систем, интересных с других точек зрения. Также делается акцент на тенденциях развития суперкомпьютерной отрасли и описываются наиболее известные проекты построения будущих экзафлопсных суперкомпьютеров.
Раздел
Параллельные программные средства и технологии
Библиографические ссылки
- Home: TOP500.
https://www.top500.org . Cited April 25, 2021.
- Graph 500: Large-Scale Benchmarks.
https://graph500.org . Cited April 25, 2021.
- Green500: TOP500.
https://www.top500.org/lists/green500 . Cited April 25, 2021.
- HPCG: TOP500.
https://www.top500.org/lists/hpcg . Cited April 25, 2021.
- A. S. Antonov, D. A. Nikitenko, and V. V. Voevodin, “Algo500 -- A New Approach to the Joint Analysis of Algorithms and Computers,” Lobachevskii J. Math. 41 (8), 1435-1443 (2020).doi 10.1134/S1995080220080041.
- Together We Are Powerful. Folding@home.
https://foldingathome.org . Cited April 25, 2021.
- Vl. V. Voevodin and A. P. Kapitonova, Methods for Describing and Classifying Computing Systems Architectures (Mosk. Gos. Univ., Moscow, 1994) [in Russian].
- V. V. Voevodin and Vl. V. Voevodin, The Parallel Computing (BHV-Petersburg, St. Petersburg, 2002) [in Russian].
- Classification of Computing Systems Architectures | PARALLEL.RU -- Information and Analytical Center for Parallel Computing.
https://parallel.ru/computers/taxonomy . Cited April 25, 2021.
- M. J. Flynn, “Very High-Speed Computing Systems,” Proc. IEEE 54 (12), 1901-1909 (1966).
doi 10.1109/PROC.1966.5273
- M. J. Flynn, “Some Computer Organizations and Their Effectiveness,” IEEE Trans. Comput. 21 (9), 948-960 (1972).
doi 10.1109/TC.1972.5009071
- Home: European Processor Initiative.
https://www.european-processor-initiative.eu . Cited April 25, 2021.
- J. A. Fisher, P. Faraboschi, and C. Young, “VLIW Processors,” in Encyclopedia of Parallel Computing (Springer, Boston, 2011), pp. 2135-2142. doi 10.1007/978-0-387-09766-4_471.
- I. V. Afanasyev, V. V. Voevodin, K. Komatsu, and H. Kobayashi, “VGL: A High-Performance Graph Processing Framework for the NEC SX-Aurora TSUBASA Vector Architecture,” J. Supercomput. (2021). doi 10.1007/s11227-020-03564-9.
- Technology Intel® Hyper-Threading.
https://www.intel.ru/content/www/ru/ru/architecture-and-technology/hyper-threading/hyper-threading-technology.html . Cited April 25, 2021 [in Russian].
- O. E. Okonta, D. Ajani, A. A. Owolabi, et al., “Performance Evaluation of Hyper Threading Technology Architecture Using Microsoft Operating System Platform,” West Afr. J. Ind. Acad. Res. 15 (1), 52-67 (2015).
- Intel Hyper Threading Performance with a Core i7 on Ubuntu 18.04 LTS.
https://www.phoronix.com/scan.php?page=article&item=intel-ht-2018&num=4 . Cited April 25, 2021.
- What is the TensorFloat-32 Precision Format? | NVIDIA Blog.
https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format . Cited April 25, 2021.
- AMD Chips Away at Intel in World’s Top 500 Supercomputers as GPU War Looms.
https://www.crn.com/news/components-peripherals/amd-chips-away-at-intel-in-world-s-top-500-supercompu-ters-as-gpu-war-looms . Cited April 25, 2021.
- Intel Server Roadmap: 14nm Cooper Lake in 2019, 10nm Ice Lake in 2020.
https://www.anandtech.com/show/13194/intel-shows-xeon-2018-2019-roadmap-cooper-lakesp-and-ice-lakesp-confirmed . Cited April 25, 2021.
- Intel® Xeon® Gold 6248 Processor (27.5M Cache, 2.50 GHz) Product Specifications.
https://ark.intel.com/content/www/us/en/ark/products/192446/intel-xeon-gold-6248-processor-27-5m-cache-2-50-ghz.html . Cited April 25, 2021.
- EPYC 7742-AMD.
https://en.wikichip.org/wiki/amd/epyc/7742 . Cited April 25, 2021.
- Epyc-Wikipedia.
https://en.wikipedia.org/wiki/Epyc . Cited April 25, 2021.
- IBM Power9-WikiChip.
https://en.wikichip.org/wiki/ibm/microarchitectures/power9/#Memory_Hierarchy . Cited April 25, 2021.
- FUJITSU Processor A64FX.
https://www.fujitsu.com/downloads/SUPER/a64fx/a64fx_datasheet.pdf . Cited April 25, 2021.
- Baikal-M--Baikal Electronics--WikiChip.
https://en.wikichip.org/wiki/baikal/baikal-m . Cited April 25, 2021.
- Y.-C. Wang, J.-K. Chen, B.-R. Li, “An Empirical Study of HPC Workloads on Huawei Kunpeng 916 Processor,” in Proc. IEEE 25th Int. Conf. on Parallel and Distributed Systems, Tianjin, China, December 4—6, 2019 (IEEE Press, New York, 2019), pp. 360-367, doi 10.1109/ICPADS47876.2019.00057.
- Vulcan-Microarchitectures-Cavium-WikiChip.
https://en.wikichip.org/wiki/cavium/microarchitectures/vulcan . Cited April 25, 2021.
- E. Calore, A. Gabbana, S. F. Schifano, and R. Tripiccione, “ThunderX2 Performance and Energy-Efficiency for HPC Workloads,” Computation 8 (1), 1-17 (2020).
doi 10.3390/computation8010020
- Hot Chips 2020: Marvell Details ThunderX3 CPUs-Up to 60 Cores Per Die, 96 Dual-Die in 2021.
https://www.anandtech.com/show/15995/hot-chips-2020-marvell-details-thunderx3 . Cited April 25, 2021.
- eMAG-Ampere-WikiChip.
https://en.wikichip.org/wiki/ampere_computing/emag . Cited April 25, 2021.
- D. Alfonso, R. Demenko, A. Kozhin, et al., “Eight-Core ’Elbrus-8C’ Processor Microarchitecture,” Voprosy Radioelektron., No. 3, 6-13 (2016).
- Chinese Processor-Supercomputer Path.
https://www.osp.ru/os/2017/01/13051592 . Cited April 25, 2021[in Russian].
- SPARC64 XIfx: Fujitsu’s Next Generation Processor for HPC.
https://www.fujitsu.com/jp/documents/products/computing/servers/unix/sparc/events/2014/20140811hotchips26.pdf . Cited April 25, 2021.
- MN-Core: Preferred Networks.
https://projects.preferred.jp/mn-core/en . Cited April 25, 2021.
- Cerebras Wafer Scale Engine.
https://www.ixbt.com/news/2019/08/20/cerebras-wafer-scale-engine-ipad-pro-1-tdp-15.html . Cited April 25, 2021 [in Russian].
- First GPU on CDNA Architecture.
https://www.hardwareluxx.ru/index.php/news/hardware/grafikkarten/50677-pervyj-gpu-na-arkhitekture-cdna-amd-predstavila-instinct-mi100.html . Cited April 25, 2021 [in Russian].
- K. Komatsu, R. Egawa, Y. Isobe, et al., “An Approach to the Highest Efficiency of the HPCG Benchmark on the SX-ACE Supercomputer,” in Proc. Int. Conf. on High Performance Computing, Networking, Storage, and Analysis, Austin, USA, November 15-20, 2015 ,
http://sc15.supercomputing.org/sites/all/themes/SC15images/tech_poster/poster_files/post277s2-file3.pdf . Cited April 15, 2021.
- K. Komatsu, S. Momose, Y. Isobe, et al., “Performance Evaluation of a Vector Supercomputer SX-Aurora TSUBASA,” in Proc. Int. Conf. for High Performance Computing, Networking, Storage, and Analysis, Dallas, USA, November 11-16, 2018 (IEEE Press, Piscataway, 2018), pp. 685-696.
doi 10.1109/SC.2018.00057
- J. Dongarra, Report on the Tianhe-2A System , Tech Report No. ICL-UT-17-04 (Univ. Tennessee, Knoxville, 2017).
https://www.dropbox.com/s/0jyh5qlgok73t1f/TH-2A-report.pdf?dl=0.
- Hardwareluxx: The Third Generation of Google TPU.
https://www.hardwareluxx.ru/index.php/news/hardware/prozessoren/44682-googles-tpu.html . Cited April 25, 2021 [in Russian].
- System Architecture: Cloud TPU.
https://cloud.google.com/tpu/docs/system-architecture . Cited April 25, 2021.
- Google Cloud Blog.
https://cloud.google.com/blog/products/gcp/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu . Cited April 25, 2021.
- What Is a Data Processing Unit (DPU).
https://www.forbes.com/sites/janakirammsv/2020/10/11/what-is-a-data-processing-unit-dpu-and-why-is-nvidia-betting-on-it . Cited April 25, 2021.
- What Is a DPU? | NVIDIA Blog.
https://blogs.nvidia.com/blog/2020/05/20/whats-a-dpu-data-processing-unit . Cited April 30, 2021.
- Hardwareluxx: NVIDIA + Mellanox.
https://www.hardwareluxx.ru/index.php/news/hardware/grafikkarten/50439-nvidia-mellanox-konvergentsiya-tekhnologij-v-budushchikh-dpu.html . Cited April 25, 2021 [in Russian].
- NVIDIA BLUEFIELD-2 DPU.
https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/documents/datasheet-nvidia-bluefield-2-dpu.pdf . Cited April 25, 2021.
- INTUIT: Lecture.
https://intuit.ru/studies/courses/1156/190/lecture/4942?page=4 . Cited April 25, 2021 [in Russian].
- NUMA Deep Dive. Part 1.
https://frankdenneman.nl/2016/07/07/numa-deep-dive-part-1-uma-numa . Cited April 25, 2021.
- D. Kanter, “The Common System Interface: Intel’s Future Interconnect,” Real World Technologies (2007).
https://www.realworldtech.com/common-system-interface/. Cited April 25, 2021.
- T. Wang, Z. Su, Y. Xia, et al., “NovaCube: A Low Latency Torus-Based Network Architecture for Data Centers,” in Proc. Global Communications Conf., Austin, USA, December 8—12, 2014 (IEEE Press, New York, 2014), pp. 2252-2257,
doi 10.1109/GLOCOM.2014.7037143
- Fat Tree -- Википедия.
https://ru.wikipedia.org/wiki/Fat_Tree . Cited April 25, 2021 [in Russian].
- X. Yuan, “On Nonblocking Folded-Clos Networks in Computer Communication Environments,” in IEEE International Parallel & Distributed Processing Symposium. (IEEE Press, Anchorage, USA, 2011), pp. 188-196,
doi 10.1109/IPDPS.2011.27
- Butterfly Network: Wikipedia.
- J. Kim, W. J. Dally, and D. Abts, “Flattened Butterfly: A Cost-Efficient Topology for High-Radix Networks,” ACM SIGARCH Computer Architecture News 35 (2) (2007). doi 10.1145/1273440.1250679.
- Intel® Omni-Path Architecture Performance Tested for HPC.
https://www.intel.ru/content/www/ru/ru/high-performance-computing-fabrics/omni-path-architecture-performance-overview.html . Cited April 25, 2021.
- Low-Latency Ethernet Solutions for High-Performance Computing.
https://www.cisco.com/c/dam/en_us/solutions/industries/docs/education/ethernet-solutions-high-performance-computing-education.pdf . Cited April 25, 2021.
- Y. Ajima, T. Kawashima, T. Okamoto, et al., “The Tofu Interconnect D,” in Proc. IEEE Int. Conf. on Cluster Computing, Belfast, UK, September 10-13, 2018 (IEEE Press, New York, 2018), pp. 646-654,
doi 10.1109/CLUSTER.2018.00090
- Y. Ajima, T. Inoue, S. Hiramoto, and T. Shimizu, “Tofu: Interconnect for the K Computer,” Fujitsu Sci. Tech. J. 48 (3), 280—285 (2012).
- Aries Network on Theta | Argonne Leadership Computing Facility.
https://www.alcf.anl.gov/support-center/theta/aries-network-theta . Cited April 25, 2021.
- S. Parker, S. Chunduri, K. Harms, and K. Kandalla, “Performance Evaluation of MPI on Cray XC40 Xeon Phi Systems,”
https://cug.org/proceedings/cug2018_proceedings/includes/files/pap131s2-file1.pdf . Cited April 25, 2021.
- K. Harms, T. Leggett, B. Allen, et al., “Theta: Rapid Installation and Acceptance of an XC40 KNL System,” Concurr. Comput. 30 (1) (2018).
doi 10.1002/cpe.4336
- HPE Cray.
https://buy.hpe.com/ru/ru/servers/cray-systems/cray-supercomputer/cray-supercomputer/hpe-cray-supercomputers/p/1012927320 . Cited April 25, 2021 [in Russian].
- D. Sensi, S. Girolamo, K. H. McMahon, et al., “An In-Depth Analysis of the Slingshot Interconnect,” in Proc. Int. Conf. for High Performance Computing, Networking, Storage and Analysis, Atlanta, USA, November 9-19, 2020 (IEEE Press, New York, 2020), pp. 481-494,
doi 10.1109/SC41405.2020.00039
- T. F. Ismagilov, A. S. Semyonov, and A. S. Simonov, “Results of Evaluation Testing of the Angara Domestic High-Speed Communication Network,” in Russian Supercomputing Days (Mosk. Gos. Univ., Moscow, 2016), pp. 626-639.
- A. Simonov, I. Zhabin, E. Kushtanov, et al., “Angara Interconnect: Architecture and Performance Results,” Voprosy Kiberbezopasn., No. 4, 46-53 (2019).
- Xilinx’s Project Everest Looks Like Bad News for Intel.
https://www.fool.com/investing/2018/04/17/xilinxs-project-everest-looks-like-bad-news-for-in.aspx . Cited April 25, 2021.
- Xilinx vs. Intel High-End FPGA Series Comparison.
https://hardwarebee.com/xilinx-vs-intel-high-end-fpga-series-comparison . Cited April 25, 2021.
- S. Craven and P. Athanas, “Examining the Viability of FPGA Supercomputing,” EURASIP J. Embed. Syst. (2007),
doi 10.1155/2007/93652
- Specifications: Supercomputer Fugaku.
https://www.fujitsu.com/global/about/innovation/fugaku/specifications . Cited April 25, 2021.
- Y. Ajima, S. Sumimoto, and T. Shimizu, “Tofu: A 6D Mesh/Torus Interconnect for Exascale Computers,” Computer 42 (11), 36-40 (2009).
doi 10.1109/MC.2009.370
- FEFS: Scalable Cluster File System.
https://www.fujitsu.com/downloads/TC/sc11/fefs-sc11.pdf . Cited April 25, 2021.
- Next Generation File System Design.
http://oss-tsukuba.org/wp-content/uploads/2018/09/2018-GFarmWS-Fujitsu.pdf . Cited April 25, 2021.
- About Fugaku: RIKEN Center for Computational Science.
https://www.r-ccs.riken.jp/en/fugaku/about . Cited April 25, 2021.
- Fact Sheet: Collaboration of Oak Ridge, Argonne, and Livermore (CORAL).
https://www.energy.gov/downloads/fact-sheet-collaboration-oak-ridge-argonne-and-livermore-coral . Cited April 25, 2021.
- Summit: Oak Ridge Leadership Computing Facility.
https://www.olcf.ornl.gov/olcf-resources/compute-systems/summit . Cited April 25, 2021.
- Sierra: High Performance Computing.
https://hpc.llnl.gov/hardware/platforms/sierra . Cited April 25, 2021.
- H. Fu, J. Liao, J. Yang, et al., “The Sunway TaihuLight Supercomputer: System and Applications,” Sci. China Inf. Sci. 59 (2016). doi 10.1007/s11432-016-5588-7.
- AI of the Storm: How We Built the Most Powerful Industrial Computer in the U.S. in Three Weeks During a Pandemic.
https://blogs.nvidia.com/blog/2020/08/14/making-selene-pandemic-ai . Cited April 25, 2021.
- Role of the New Machine: Amid Shutdown, NVIDIA’s Selene Supercomputer Busier Than Ever.
https://blogs.nvidia.com/blog/2020/12/18/nvidia-selene-busy . Cited April 25, 2021.
- Forschungszentrum Jülich: JUWELS.
https://www.fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JUWELS/JUWELS_node.html . Cited April 25, 2021.
- HPC5: the Supercomputer Working for Energy.
https://www.eni.com/en-IT/operations/green-data-center-hpc5.html . Cited April 25, 2021.
- FRONTERA.
https://frontera-portal.tacc.utexas.edu . Cited April 25, 2021.
- Texas Advanced Computing Center: TACC LAUNCHES EXPANDED FRONTERA SUPERCOMPUTER TO SUPPORT URGENT COMPUTING.
https://www.tacc.utexas.edu/-/tacc-launches-expanded-frontera-supercomputer-to-support-urgent-computing . Cited April 25, 2021.
- Aramco and STC Unveil Dammam 7 Supercomputer.
https://www.aramco.com/en/news-media/news/2021/aramco-and-stc-unveil-dammam-7-supercomputer . Cited April 25, 2021.
- NVIDIA DGX SuperPOD for Enterprise.
https://www.nvidia.com/en-us/data-center/dgx-superpod . Cited April 25, 2021.
- Green500 Supercomputer Delivers the World’s Best Performance-Per-Watt.
https://www.supermicro.com/en/success-story/green500-pfn-number1 . Cited April 25, 2021.
- Preferred Networks’ MN-3 Supercomputer.
https://www.preferred.jp/en/news/pr20201117 . Cited April 25, 2021.
- SberCloud: Christofari.
https://sbercloud.ru/ru/christofari . Cited April 25, 2021 [in Russian].
- The Christofari Supercomputer.
https://tass.ru/ekonomika/8121173 . Cited April 25, 2021 [in Russian].
- V. Voevodin, A. Antonov, D. Nikitenko, et al., “Lomonosov-2: Petascale Supercomputing at Lomonosov Moscow State University,” in Contemporary High Performance Computing: from Petascale toward Exascale (CRC Press, Boca Raton, 2019), Vol. 3, pp. 305-330.
- V. V. Voevodin, A. S. Antonov, D. A. Nikitenko, et al., “Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community,” Supercomput. Front. Innov. 6 (2), 4-11 (2019).
doi 10.14529/jsfi190201
- PARALLEL.RU: Supercomputer Lomonosov-2.
https://parallel.ru/cluster/lomonosov2.html . Cited April 25, 2021 [in Russian].
- Blue Waters User Portal.
https://bluewaters.ncsa.illinois.edu . Cited April 25, 2021.
- D. E. Shaw, J. P. Grossman, J. A. Bank, et al., “Anton 2: Raising the Bar for Performance and Programmability in a Special-Purpose Molecular Dynamics Supercomputer,” in Proc. Int. Conf. for High Performance Computing, Networking, Storage, and Analysis, New Orleans, USA, November 16-21, 2014 (IEEE Press, Piscataway, 2014), pp. 41-53, doi 10.1109/SC.2014.9.
- ARM’s Secret Recipe for Power Efficient Processing.
https://www.androidauthority.com/arms-secret-recipe-for-power-efficient-processing-409850 . Cited April 25, 2021.
- EPYC: A Study in Energy Efficient CPU Design.
https://www.amd.com/system/files/documents/The-Energy-Efficient-AMD-EPYC-Design.pdf . Cited April 25, 2021.
- Arm Supercomputer Captures The Energy Efficiency Crown.
https://www.nextplatform.com/2019/11/22/arm-supercomputer-captures-the-energy-efficiency-crown . Cited April 25, 2021.
- K. J. Barker, K. Davis, A. Hoisie, et al., “Entering the Petaflop Era: The Architecture and Performance of Roadrunner,” in Proc. 2008 ACM/IEEE Conference on Supercomputing, Austin, USA, November 15-21, 2008 (IEEE Press, Austin, 2008), pp. 1-11, doi 10.1109/SC.2008.5217926.1
- P. Thibodeau, “Scientists, IT Community Await Exascale Computers,” (2009).
https://www.computerworld.com/article/2550451/scientists-it-community-await-exascale-computers.html . Cited April 25, 2021.1
- Folding@Home Network Breaks the ExaFLOP Barrier in Fight Against Coronavirus.
https://www.tomshardware.com/news/folding-at-home-breaks-exaflop-barrier-fight-coronavirus-covid-19 . Cited April 25, 2021.1
- P. Thibodeau, “Obama Sets 126M for Next-Gen Supercomputing’’ (2011).
https://www.computerworld.com/article/2513219/obama-sets-126m-for-next-gen-supercomputing.html . Cited April 25, 2021.1
- Frontier.
https://www.olcf.ornl.gov/frontier . Cited April 25, 2021.1
- Aurora: Argonne Leadership Computing Facility.
https://alcf.anl.gov/aurora . Cited April 25, 2021.1
- B. D. Zarley, “America’s First Exascale Supercomputer to be Built by 2021,” (2019).
https://www.theverge.com/2019/3/18/18271328/supercomputer-build-date-exascale-intel-argonne-national-laboratory-energy . Cited April 25, 2021.1
- HPE Slingshot Interconnect: High Performance Network for HPE Cray Supercomputers.
https://www.hpe.com/us/en/compute/hpc/slingshot-interconnect.html . Cited April 25, 2021.1
- El Capitan Supercomputer at Lawrence Livermore National Lab.
https://www.hpe.com/us/en/compute/hpc/cray/doe-el-capitan-press-release.html . Cited April 25, 2021.1
- China Fleshes Out Exascale Design for Tianhe-3 Supercomputer.
https://www.nextplatform.com/2019/05/02/china-fleshes-out-exascale-design-for-tianhe-3 . Cited April 25, 2021.1
- China Launches Exascale Supercomputer Prototype.
http://www.xinhuanet.com/english/2018-08/06/c_137369865.htm . Cited April 25, 2021.1
- China Launches Third Prototype Exascale Computer.
http://www.xinhuanet.com/english/2018-10/22/c_137550589.htm . Cited April 25, 2021.1
- HPC AI500: A Benchmark Suite for HPC AI Systems.
https://www.benchcouncil.org/HPCAI500 . Cited April 25, 2021.1
- Japan’s Fugaku Supercomputer.
https://www.japantimes.co.jp/news/2021/01/07/business/tech/japans-fugaku-supercomputer . Cited April 25, 2021.1
- CRESTA: Developing Techniques and Solutions.
http://www.cresta-project.eu . Cited April 25, 2021.1
- DEEP Projects.
https://www.deep-projects.eu . Cited April 25, 2021.1
- Home: Mont-Blanc.
https://www.montblanc-project.eu . Cited April 25, 2021.1
- Home: European High Performance Computer Joint Undertaking.
https://eurohpc-ju.europa.eu . Cited April 25, 2021.1
- LUMI Front Page.
https://www.lumi-supercomputer.eu . Cited April 25, 2021.1
- MareNostrum.
https://www.bsc.es/marenostrum . Cited April 25, 2021.1
- Leonardo Pre-Exascale Supercomputer.
https://www.cineca.it/en/hot-topics/Leonardo . Cited April 25, 2021.1
- NVIDIA Extends Data Center Infrastructure Processing Roadmap with BlueField-3 | NVIDIA Newsroom.
https://nvidianews.nvidia.com/news/nvidia-extends-data-center-infrastructure-processing-roadmap-with-bluefield-3 . Cited April 25, 2021.