CXL3.1，CXLSpecification-rev3p1-ver1p0-2023August9

PCIE

需积分: 5 74 浏览量 2024-04-23 11:11:30 上传评论收藏 11.52MB PDF 举报

资源推荐

资源详情

资源评论

Compute Express Link

(CXL

)

Specification

August 2023

Revision 3.1

- 1 -

C6124.0003 BN/FCURCI 39895222v1

LEGAL NOTICE FOR THIS PUBLICLY-AVAILABLE SPECIFICATION FROM COMPUTE EXPRESS LINK CONSORTIUM, INC.

This CXL Specification Revision 3.1 (this “CXL Specification” or this “document”) is owned by and is proprietary to Compute Express Link

Consortium, Inc., a Delaware nonprofit corporation (sometimes referred to as “CXL” or the “CXL Consortium” or the “Company”) and/or its

successors and assigns.

NOTICE TO USERS WHO ARE MEMBERS OF THE CXL CONSORTIUM:

If you are a Member of the CXL Consortium (sometimes referred to as a “CXL Member”), and even if you have received this publicly-available version

of this CXL Specification after agreeing to CXL Consortium’s Evaluation Copy Agreement (a copy of which is available

https://www.computeexpresslink.org/download-the-specification, each such CXL Member must also be in compliance with all of the following CXL

Consortium documents, policies and/or procedures (collectively, the “CXL Governing Documents”) in order for such CXL Member’s use and/or

implementation of this CXL Specification to receive and enjoy all of the rights, benefits, privileges and protections of CXL Consortium membership: (i)

CXL Consortium’s Intellectual Property Policy; (ii) CXL Consortium’s Bylaws; (iii) any and all other CXL Consortium policies and procedures; and (iv)

the CXL Member’s Participation Agreement.

NOTICE TO NON-MEMBERS OF THE CXL CONSORTIUM:

If you are not a CXL Member and have received this publicly-available version of this CXL Specification, your use of this document is subject to your

compliance with, and is limited by, all of the terms and conditions of the CXL Consortium’s Evaluation Copy Agreement (a copy of which is available at

https://www.computeexpresslink.org/download-the-specification).

In addition to the restrictions set forth in the CXL Consortium’s Evaluation Copy Agreement, any references or citations to this document must

acknowledge the Compute Express Link Consortium, Inc.’s sole and exclusive copyright ownership of this CXL Specification. The proper copyright

any such citation or reference to this document you are not permitted to revise, alter, modify, make any derivatives of, or otherwise amend the referenced

portion of this document in any way without the prior express written permission of the Compute Express Link Consortium, Inc.

Except for the limited rights explicitly given to a non-CXL Member pursuant to the explicit provisions of the CXL Consortium’s Evaluation Copy

Agreement which governs the publicly-available version of this CXL Specification, nothing contained in this CXL Specification shall be deemed as

granting (either expressly or impliedly) to any party that is not a CXL Member: (ii) any kind of license to implement or use this CXL Specification or any

portion or content described or contained therein, or any kind of license in or to any other intellectual property owned or controlled by the CXL

Consortium, including without limitation any trademarks of the CXL Consortium.; or (ii) any benefits and/or rights as a CXL Member under any CXL

Governing Documents.

LEGAL DISCLAIMERS FOR ALL PARTIES:

THIS DOCUMENT AND ALL SPECIFICATIONS AND/OR OTHER CONTENT PROVIDED HEREIN IS PROVIDED ON AN “AS IS” BASIS. TO

THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, COMPUTE EXPRESS LINK CONSORTIUM, INC. (ALONG WITH THE

CONTRIBUTORS TO THIS DOCUMENT) HEREBY DISCLAIM ALL REPRESENTATIONS, WARRANTIES AND/OR COVENANTS, EITHER

EXPRESS OR IMPLIED, STATUTORY OR AT COMMON LAW, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, VALIDITY, AND/OR NON-INFRINGEMENT.

In the event this CXL Specification makes any references (including without limitation any incorporation by reference) to another standard’s setting

organization’s or any other party’s (“Third Party”) content or work, including without limitation any specifications or standards of any such Third Party

(“Third Party Specification”), you are hereby notified that your use or implementation of any Third Party Specification: (i) is not governed by any of

the CXL Governing Documents; (ii) may require your use of a Third Party’s patents, copyrights or other intellectual property rights, which in turn may

require you to independently obtain a license or other consent from that Third Party in order to have full rights to implement or use that Third Party

Specification; and/or (iii) may be governed by the intellectual property policy or other policies or procedures of the Third Party which owns the Third

Party Specification. Any trademarks or service marks of any Third Party which may be referenced in this CXL Specification is owned by the respective

owner of such marks.

NOTICE TO ALL PARTIES REGARDING THE PCI-SIG UNIQUE VALUE PROVIDED IN THIS CXL SPECIFICATION:

NOTICE TO USERS: THE UNIQUE VALUE THAT IS PROVIDED IN THIS CXL SPECIFICATION IS FOR USE IN VENDOR DEFINED

MESSAGE FIELDS, DESIGNATED VENDOR SPECIFIC EXTENDED CAPABILITIES, AND ALTERNATE PROTOCOL NEGOTIATION ONLY

AND MAY NOT BE USED IN ANY OTHER MANNER, AND A USER OF THE UNIQUE VALUE MAY NOT USE THE UNIQUE VALUE IN A

MANNER THAT (A) ALTERS, MODIFIES, HARMS OR DAMAGES THE TECHNICAL FUNCTIONING, SAFETY OR SECURITY OF THE PCI-

SIG ECOSYSTEM OR ANY PORTION THEREOF, OR (B) COULD OR WOULD REASONABLY BE DETERMINED TO ALTER, MODIFY,

HARM OR DAMAGE THE TECHNICAL FUNCTIONING, SAFETY OR SECURITY OF THE PCI-SIG ECOSYSTEM OR ANY PORTION

THEREOF (FOR PURPOSES OF THIS NOTICE, “PCI-SIG ECOSYSTEM” MEANS THE PCI-SIG SPECIFICATIONS, MEMBERS OF PCI-SIG

AND THEIR ASSOCIATED PRODUCTS AND SERVICES THAT INCORPORATE ALL OR A PORTION OF A PCI-SIG SPECIFICATION AND

EXTENDS TO THOSE PRODUCTS AND SERVICES INTERFACING WITH PCI-SIG MEMBER PRODUCTS AND SERVICES).

Contents

August 7, 2023 Compute Express Link Specification

Revision 3.1, Version 1.0 3

Contents

1.0 Introduction ............................................................................................................ 48

1.1 Audience.......................................................................................................... 48

1.2 Terminology/Acronyms ...................................................................................... 48

1.3 Reference Documents ........................................................................................ 59

1.4 Motivation and Overview .................................................................................... 60

1.4.1 CXL................................................................................................... 60

1.4.2 Flex Bus ............................................................................................ 63

1.5 Flex Bus Link Features ....................................................................................... 65

1.6 Flex Bus Layering Overview ................................................................................ 65

1.7 Document Scope ............................................................................................... 66

2.0 CXL System Architecture ......................................................................................... 69

2.1 CXL Type 1 Device ............................................................................................ 70

2.2 CXL Type 2 Device ............................................................................................ 70

2.2.1 Back-Invalidate Snoop Coherence for HDM-DB........................................ 71

2.2.2 Bias-based Coherency Model for HDM-D Memory..................................... 71

2.2.2.1 Host Bias .......................................................................... 72

2.2.2.2 Device Bias ....................................................................... 72

2.2.2.3 Mode Management ............................................................. 73

2.3 CXL Type 3 Device ............................................................................................ 74

2.4 Multi Logical Device (MLD).................................................................................. 75

2.4.1 LD-ID for CXL.io and CXL.mem ............................................................. 75

2.4.1.1 LD-ID for CXL.mem ............................................................ 75

2.4.1.2 LD-ID for CXL.io ................................................................ 75

2.4.2 Pooled Memory Device Configuration Registers ....................................... 76

2.4.3 Pooled Memory and Shared FAM ........................................................... 77

2.4.4 Coherency Models for Shared FAM......................................................... 77

2.5 Multi-Headed Device.......................................................................................... 79

2.5.1 LD Management in MH-MLDs ................................................................ 80

2.6 CXL Device Scaling ............................................................................................ 80

2.7 CXL Fabric........................................................................................................ 81

2.8 Global FAM (G-FAM) Type 3 Device ...................................................................... 81

2.9 Manageability Overview ..................................................................................... 81

3.0 CXL Transaction Layer ............................................................................................. 83

3.1 CXL.io.............................................................................................................. 83

3.1.1 CXL.io Endpoint .................................................................................. 84

3.1.2 CXL Power Management VDM Format .................................................... 84

3.1.2.1 Credit and PM Initialization .................................................. 88

3.1.3 CXL Error VDM Format......................................................................... 89

3.1.4 Optional PCIe Features Required for CXL ................................................ 90

3.1.5 Error Propagation................................................................................ 90

3.1.6 Memory Type Indication on ATS............................................................ 91

3.1.7 Deferrable Writes................................................................................ 91

3.1.8 PBR TLP Header (PTH) ......................................................................... 91

3.1.8.1 Transmitter Rules Summary ................................................ 92

3.1.8.2 Receiver Rules Summary..................................................... 92

3.1.9 VendPrefixL0 ...................................................................................... 94

3.1.10 CXL DevLoad (CDL) Field in UIO Completions.......................................... 95

3.1.11 CXL Fabric-related VDMs...................................................................... 95

3.1.11.1 Host Management Transaction Flows of GFD .......................... 97

3.1.11.2 Downstream Proxy Command (DPCmd) VDM ......................... 99

3.1.11.3 Upstream Command Pull (UCPull) VDM ............................... 100

Contents

August 7, 2023 Compute Express Link Specification

Revision 3.1, Version 1.0 4

3.1.11.4 Downstream Command Request

(DCReq, DCReq-Last, DCReq-Fail) VDMs ............................. 101

3.1.11.5 Upstream Command Response

(UCRsp, UCRsp-Last, UCRsp-Fail) VDMs .............................. 102

3.1.11.6 GFD Async Message (GAM) VDM ........................................ 102

3.1.11.7 Route Table Update (RTUpdate) VDM.................................. 103

3.1.11.8 Route Table Update Response

(RTUpdateAck, RTUpdateNak) VDMs ................................... 103

3.2 CXL.cache ...................................................................................................... 104

3.2.1 Overview ......................................................................................... 104

3.2.2 CXL.cache Channel Description ........................................................... 105

3.2.2.1 Channel Ordering ............................................................. 105

3.2.2.2 Channel Crediting............................................................. 105

3.2.3 CXL.cache Wire Description ................................................................ 106

3.2.3.1 D2H Request ................................................................... 106

3.2.3.2 D2H Response ................................................................. 107

3.2.3.3 D2H Data........................................................................ 108

3.2.3.4 H2D Request ................................................................... 108

3.2.3.5 H2D Response ................................................................. 109

3.2.3.6 H2D Data........................................................................ 110

3.2.4 CXL.cache Transaction Description ...................................................... 110

3.2.4.1 Device-attached Memory Flows for HDM-D/HDM-DB ............. 110

3.2.4.2 Device to Host Requests ................................................... 111

3.2.4.3 Device to Host Response................................................... 123

3.2.4.4 Host to Device Requests ................................................... 124

3.2.4.5 Host to Device Response................................................... 125

3.2.5 Cacheability Details and Request Restrictions........................................ 127

3.2.5.1 GO-M Responses.............................................................. 127

3.2.5.2 Device/Host Snoop-GO-Data Assumptions ........................... 127

3.2.5.3 Device/Host Snoop/WritePull Assumptions........................... 127

3.2.5.4 Snoop Responses and Data Transfer on CXL.cache Evicts ...... 128

3.2.5.5 Multiple Snoops to the Same Address ................................. 128

3.2.5.6 Multiple Reads to the Same Cacheline ................................. 128

3.2.5.7 Multiple Evicts to the Same Cacheline ................................. 128

3.2.5.8 Multiple Write Requests to the Same Cacheline .................... 128

3.2.5.9 Multiple Read and Write Requests to the Same Cacheline ...... 128

3.2.5.10 Normal Global Observation (GO) ........................................ 129

3.2.5.11 Relaxed Global Observation (FastGO).................................. 129

3.2.5.12 Evict to Device-attached Memory ....................................... 129

3.2.5.13 Memory Type on CXL.cache ............................................... 129

3.2.5.14 General Assumptions ........................................................ 129

3.2.5.15 Buried Cache State Rules .................................................. 130

3.2.5.16 H2D Req Targeting Device-attached Memory ....................... 131

3.3 CXL.mem ....................................................................................................... 132

3.3.1 Introduction ..................................................................................... 132

3.3.2 CXL.mem Channel Description ............................................................ 133

3.3.2.1 Direct P2P CXL.mem for Accelerators .................................. 134

3.3.2.2 Snoop Handling with Direct P2P CXL.mem ........................... 134

3.3.3 Back-Invalidate Snoop....................................................................... 135

3.3.4 QoS Telemetry for Memory................................................................. 136

3.3.4.1 QoS Telemetry Overview................................................... 136

3.3.4.2 Reference Model for Host/Peer Support of QoS Telemetry ...... 137

3.3.4.3 Memory Device Support for QoS Telemetry.......................... 138

3.3.5 M2S Request (Req) ........................................................................... 148

3.3.6 M2S Request with Data (RwD) ............................................................ 152

3.3.6.1 Trailer Present for RwD (256B Flit) ..................................... 154

3.3.7 M2S Back-Invalidate Response (BIRsp)................................................ 154

3.3.8 S2M Back-Invalidate Snoop (BISnp) .................................................... 155

Contents

August 7, 2023 Compute Express Link Specification

Revision 3.1, Version 1.0 5

3.3.8.1 Rules for Block Back-Invalidate Snoops ............................... 156

3.3.9 S2M No Data Response (NDR) ............................................................ 157

3.3.10 S2M Data Response (DRS) ................................................................. 158

3.3.10.1 Trailer Present for DRS (256B Flit)...................................... 159

3.3.11 Forward Progress and Ordering Rules .................................................. 160

3.3.11.1 Buried Cache State Rules for HDM-D/HDM-DB...................... 160

3.4 Transaction Ordering Summary ......................................................................... 162

3.5 Transaction Flows to Device-attached Memory .................................................... 165

3.5.1 Flows for Back-Invalidate Snoops on CXL.mem ..................................... 165

3.5.1.1 Notes and Assumptions..................................................... 165

3.5.1.2 BISnp Blocking Example ................................................... 166

3.5.1.3 Conflict Handling.............................................................. 167

3.5.1.4 Block Back-Invalidate Snoops ............................................ 169

3.5.2 Flows for Type 1 Devices and Type 2 Devices........................................ 171

3.5.2.1 Notes and Assumptions..................................................... 171

3.5.2.2 Requests from Host .......................................................... 172

3.5.2.3 Requests from Device in Host and Device Bias ..................... 179

3.5.3 Type 2 Memory Flows and Type 3 Memory Flows................................... 184

3.5.3.1 Speculative Memory Read ................................................. 184

3.6 Flows to HDM-H in a Type 3 Device.................................................................... 185

4.0 CXL Link Layers ..................................................................................................... 187

4.1 CXL.io Link Layer ............................................................................................ 187

4.2 CXL.cache and CXL.mem 68B Flit Mode Common Link Layer ................................. 188

4.2.1 Introduction ..................................................................................... 188

4.2.2 High-Level CXL.cachemem Flit Overview .............................................. 190

4.2.3 Slot Format Definition........................................................................ 197

4.2.3.1 H2D and M2S Formats ...................................................... 198

4.2.3.2 D2H and S2M Formats ...................................................... 203

4.2.4 Link Layer Registers .......................................................................... 208

4.2.5 68B Flit Packing Rules ....................................................................... 208

4.2.6 Link Layer Control Flit........................................................................ 210

4.2.7 Link Layer Initialization...................................................................... 214

4.2.8 CXL.cachemem Link Layer Retry ......................................................... 215

4.2.8.1 LLR Variables................................................................... 215

4.2.8.2 LLCRD Forcing ................................................................. 217

4.2.8.3 LLR Control Flits............................................................... 218

4.2.8.4 RETRY Framing Sequences ................................................ 219

4.2.8.5 LLR State Machines .......................................................... 220

4.2.8.6 Interaction with vLSM Retrain State.................................... 224

4.2.8.7 CXL.cachemem Flit CRC .................................................... 224

4.2.9 Viral................................................................................................ 226

4.3 CXL.cachemem Link Layer 256B Flit Mode .......................................................... 226

4.3.1 Introduction ..................................................................................... 226

4.3.2 Flit Overview .................................................................................... 226

4.3.3 Slot Format Definition........................................................................ 231

4.3.3.1 Implicit Data Slot Decode.................................................. 245

4.3.3.2 Trailer Decoder ................................................................ 246

4.3.4 256B Flit Packing Rules...................................................................... 247

4.3.5 Credit Return ................................................................................... 249

4.3.6 Link Layer Control Messages............................................................... 252

4.3.6.1 Link Layer Initialization ..................................................... 254

4.3.6.2 Viral Injection and Containment ......................................... 254

4.3.6.3 Late Poison ..................................................................... 255

4.3.6.4 Link Integrity and Data Encryption (IDE) ............................. 256

4.3.7 Credit Return Forcing ........................................................................ 256

4.3.8 Latency Optimizations ....................................................................... 256

剩余1165页未读，继续阅读

评论收藏

内容反馈

qq_34694359

粉丝: 1
资源: 4

CXL3.1 ， CXL Specification-rev3p1-ver1p0-2023August9

最新资源

CXL3.1 ， CXL Specification-rev3p1-ver1p0-2023August9

Compute Express Link（CXL） 3.0 Specification

国内小芯片接口总线技术标准，2023年最新发布

PCIe Base SPEC v.3.1

cxl 3.1-CXL Specification-rev3p1-ver1p0-2023August9-watermark

CXL 3.1 SPEC资源

cxl2.0协议-CXL-2.0-Specification-r1.0-2020Oct26

CXL-FMS 2023 Tutorial-FINAL

CXL3.1第7章节翻译

CXL spec 3.0 Compute Express LInk 3.0

UCIe-Rev1p0-Consortium-Feb24th-2022-Final

Compute EXpress Link（CXL）3.0规格说明书

CXL1.1协议规范文档-中文版

CXL2.0协议规范文档-中文版

SATA SPEC 3.1

CXL 2.0 PPT学习资源

CXL Fabric Manager API over MCTP Binding Specification

cxl_spec_v1p1.pdf

JEDEC JESD405-1-2023 Compute Express Link (CXL) Memory Module

CXL 2.0 Spec文档

CXL1.1协议规范文档

cxl1.1协议的中文翻译版

cxl 要用 自己过来拿啊

自动控制系统：3.1 V-M系统的可逆线路和工作状态.ppt

可靠稳定和高效的LED日光灯驱动芯片CXL9910

伟肯CX/CXL/CXS系列中文手册说明书

CXL_white_paper.pdf

CXL 1.1 PPT学习资源

PCI Express 5.0 Specification

CXL2.0协议规范资料

CXL Type 3 Device Component Command Interface over MCTP Binding

最新资源

cxl 要用自己过来拿啊