-
Notifications
You must be signed in to change notification settings - Fork 4
Expand file tree
/
Copy pathconfig_files.html
More file actions
307 lines (303 loc) · 12.6 KB
/
config_files.html
File metadata and controls
307 lines (303 loc) · 12.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" type="text/css" href="styles/style.css">
<title>OpenFAM: A library for programming Fabric-Attached Memory</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<header>
<h1>OpenFAM Reference Implementation</h1>
</header>
<section>
<nav>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="release_notes.html">Release Notes</a></li>
<li><a href="limitations.html">Design Choices</a></li>
<li><a href="errors.html">Exceptions and Error Codes</a></li>
<li><a href="services.html">Services</a></li>
<li><a href="config_files.html">Configuration Files</a></li>
</ul>
<hr>
Initialization and Finalization
<ul>
<li><a href="fam_initialize.html">fam_initialize</a></li>
<li><a href="fam_finalize.html">fam_finalize</a></li>
<li><a href="fam_abort.html">fam_abort</a></li>
</ul>
<hr>
Options & Query
<ul>
<li><a href="fam_list_options.html">fam_list_options</a></li>
<li><a href="fam_get_option.html">fam_get_option</a></li>
<li><a href="fam_lookup.html">fam_lookup</a></li>
<li><a href="fam_stat.html">fam_stat</a></li>
</ul>
<hr>
Memory Allocation
<ul>
<li><a href="fam_create_region.html">fam_create_region</a></li>
<li><a href="fam_destroy_region.html">fam_destroy_region</a></li>
<li><a href="fam_resize_region.html">fam_resize_region</a></li>
<li><a href="fam_allocate.html">fam_allocate</a></li>
<li><a href="fam_deallocate.html">fam_deallocate</a></li>
<li><a href="fam_change_permissions.html">fam_change_permissions</a></li>
<li><a href="fam_close.html">fam_close</a></li>
</ul>
<hr>
Memory Map
<ul>
<li><a href="fam_map.html">fam_map</a></li>
<li><a href="fam_unmap.html">fam_unmap</a></li>
</ul>
<hr>
Data Path Operations
<ul>
<li><a href="fam_get.html">fam_get</a></li>
<li><a href="fam_put.html">fam_put</a></li>
<li><a href="fam_gather.html">fam_gather</a></li>
<li><a href="fam_scatter.html">fam_scatter</a></li>
<li><a href="fam_copy.html">fam_copy</a></li>
<li><a href="fam_copy_wait.html">fam_copy_wait</a></li>
<li><a href="fam_backup.html">fam_backup</a></li>
<li><a href="fam_backup_wait.html">fam_backup_wait</a></li>
<li><a href="fam_restore.html">fam_restore</a></li>
<li><a href="fam_restore_wait.html">fam_restore_wait</a></li>
<li><a href="fam_delete_backup.html">fam_delete_backup</a></li>
<li><a href="fam_delete_backup_wait.html">fam_delete_backup_wait</a></li>
</ul>
<hr>
Atomics
<ul>
<li><a href="fam_set.html">fam_set</a></li>
<li><a href="fam_add.html">fam_add</a></li>
<li><a href="fam_subtract.html">fam_subtract</a></li>
<li><a href="fam_min.html">fam_min</a></li>
<li><a href="fam_max.html">fam_max</a></li>
<li><a href="fam_and.html">fam_and</a></li>
<li><a href="fam_or.html">fam_or</a></li>
<li><a href="fam_xor.html">fam_xor</a></li>
<li><a href="fam_fetch_TYPE.html">fam_fetch_TYPE</a></li>
<li><a href="fam_swap.html">fam_swap</a></li>
<li><a href="fam_compare_swap.html">fam_compare_swap</a></li>
<li><a href="fam_fetch_add.html">fam_fetch_add</a></li>
<li><a href="fam_fetch_subtract.html">fam_fetch_subtract</a></li>
<li><a href="fam_fetch_min.html">fam_fetch_min</a></li>
<li><a href="fam_fetch_and.html">fam_fetch_and</a></li>
<li><a href="fam_fetch_max.html">fam_fetch_max</a></li>
<li><a href="fam_fetch_or.html">fam_fetch_or</a></li>
<li><a href="fam_fetch_xor.html">fam_fetch_xor</a></li>
</ul>
<hr>
Ordering
<ul>
<li><a href="fam_barrier_all.html">fam_barrier_all</a></li>
<li><a href="fam_fence.html">fam_fence</a></li>
<li><a href="fam_quiet.html">fam_quiet</a></li>
<li><a href="fam_progress.html">fam_progress</a></li>
<li><a href="fam_context_open.html">fam_context_open</a></li>
<li><a href="fam_context_close.html">fam_context_close</a></li>
</ul>
<hr>
</nav>
<article>
<h1>Configuration Files</h1>
<p>The OpenFAM environment uses four separate configuration files, three for the memory and metadata services, and one for the OpenFAM client.
<ol>
<li><b>fam_client_interface_config.yaml</b>: Configures the client interface service.</li>
<li><b>fam_memoryserver_config.yaml</b>: Configures the memory servers.</li>
<li><b>fam_metadata_config.yaml</b>: Configures the metadata management service.</li>
<li><b>fam_pe_config.yaml</b>: Configures the OpenFAM clients.</li>
</ol>
These configuration files contain the following options.
<p>1. fam_client_interface_config.yaml
<table>
<tbody>
<tr><th>Configuration Name</th><th>Description</th></tr>
<tr><td class="code">memsrv_interface_type</td>
<td>Memory server interface type used by OpenFAM. Default value is "direct". Other option is "rpc".
</td>
</tr>
<tr><td class="code">metadata_interface_type</td>
<td>Metadata server interface type used by OpenFAM. Default value is "direct". Other option is "rpc".
</td>
</tr>
<tr><td class="code">rpc_interface</td>
<td> IP address/hostname of Client Interface Service, followed by port number.
</td>
</tr>
<tr><td class="code">memsrv_list</td>
<td>List of memory servers. Applicable only if memsrv_interface_type is rpc. Each list entry is given
in this format:<p> <memsrv_id>:<host_address>:<port_no>
</td>
</tr>
<tr><td class="code">metadata_list</td>
<td>List of metadata servers. Applicable only if metadata_interface_type is rpc. Each list entry is given
in this format:<p> <metadataserver_id>:<host_address>:<port_no>
</td>
</tr>
<tr><td class="code">rpc_framework_type</td>
<td> The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium".
</td>
</tr>
<tr><td class="code">provider</td>
<td> If "thallium" option is chosen for rpc_framework_type, it is essential to provide details of provider. Default value is "cxi". Other option is "sockets".
</td>
</tr>
</tbody>
</table>
<p>2. fam_memoryserver_config.yaml
<table>
<tbody>
<tr><th>Configuration Name</th><th>Description</th></tr>
<tr><td class="code">provider</td>
<td>Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets".
Other supported options are "cxi", "psm2" and "verbs".
</td>
</tr>
<tr><td class="code">delayed_free_threads</td>
<td>Number of threads used for delayed free.
</td>
</tr>
<tr><td class="code">ATL_threads</td>
<td> Atomic library threads count. Default value is 0, which indicates ATL is disabled.
</td>
</tr>
<tr><td class="code">ATL_queue_size</td>
<td> ATL queue size. Number of elements in each queue. Default value is 1000.
</td>
</tr>
<tr><td class="code">ATL_data_size</td>
<td> ATL data size per thread(MiB). This is used to allocate buffers and other queue related data structures.
Default data size is 1024.
</td>
</tr>
<tr><td class="code">fam_backup_path</td>
<td> Absolute path used by the memory server to store archives.
Default value is config file location.
</td>
</tr>
<tr><td class="code">memory_type</td>
<td> Type of memory used by the memory server to host FAM.
Options are "volatile" or "persistent".
Default value is "volatile".
</td>
</tr>
<tr><td class="code">fam_path</td>
<td> Absolute path to the NVMM data files used by the memory server.
Default value is an empty string.
</td>
</tr>
<tr><td class="code">rpc_interface</td>
<td> IP address/hostname of memory server, followed by port number.
Default value is an "127.0.0.1:8789".
</td>
</tr>
<tr><td class="code">resource_release</td>
<td> This option is to enable or disable resource relinquishment in FAM.
Default value is "enable".
</td>
</tr>
<tr><td class="code">libfabric_port</td>
<td>Libfabric port to be used for data path operations.
</td>
<tr><td class="code">if_device</td>
<td>NIC name to be used with memory server.
Default value is "eth0".
</td>
</tr>
<tr><td class="code">rpc_framework_type</td>
<td> The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium".
</td>
</tr>
<tr><td class="code">provider</td>
<td> The control path operations can leverage thallium libraries instead of grpc. If "thallium" option is chosen for rpc_framework_type, it is essential to provide provider details. Default value is "cxi". Other option is "sockets".
</td>
</tr>
</tr>
</tbody>
</table>
<p>3. fam_metadata_config.yaml
<table>
<tbody>
<tr><th>Configuration Name</th><th>Description</th></tr>
<tr><td class="code">metadata_manager</td>
<td> Metadata manager type used by metadata server. Default is radixtree. Currently radixtree is the only supported KVS.
</td>
</tr>
<tr><td class="code">metadata_path</td>
<td> Absolute path where metadata info is stored. By default, it will be placed in /dev/shm/<metadata_manager>.
Example: If metadata manager is radixtree, then metadata info would be placed under /dev/shm/radixtree folder.
</td>
</tr>
<tr><td class="code">enable_region_spanning</td>
<td> Flag to enable or disable region spanning. By default, it is set to true.
</td>
</tr>
<tr><td class="code">region_span_size_per_memoryserver</td>
<td> Region size per memory server (Bytes). Used only if region spanning is enabled.
Default value is 1073741824.
</td>
<tr><td class="code">dataitem_interleave_size</td>
<td> Data interleaving refers to interspersing portions of the data across the memory servers in a specified order or other arrangement. dataitem_interleave_size decides the number of bytes after which data item needs to be striped so that it gets distributed across several memory servers. Default value is "1048576".</td>
</tr>
<tr><td class="code">rpc_framework_type</td>
<td> The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium".
</td>
</tr>
<tr><td class="code">provider</td>
<td> If "thallium" option is chosen for rpc_framework_type, it is essential to provide provider details. Default value is "cxi". Other option is "sockets".
</td>
</tr>
</tbody>
</table>
<p>4. fam_pe_config.yaml
<table>
<tbody>
<tr><th>Configuration Name</th><th>Description</th></tr>
<tr><td class="code">FamThreadModel</td>
<td> Fam thread model used by OpenFAM. Default value is "single". OpenFAM supports single and multi-threaded model.
</td>
</tr>
<tr><td class="code">provider</td>
<td> Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets".
Other supported options are "cxi", "psm2" and "verbs".
</td>
</tr>
<tr><td class="code">runtime</td>
<td> Runtime option used for OpenFAM. Default value is "PMIX". Other supported options are "PMI2" and "NONE"
</td>
</tr>
<tr><td class="code">client_interface_type</td>
<td> Client interface type used by OpenFAM. Default value is "direct". Other option is "rpc"
</td>
</tr>
<tr><td class="code">fam_client_interface_service_address</td>
<td> Address of Client Interface Service. Value is specified in this format: <ip_address>:<port_no>.
Applicable only if client_interface_type is rpc. Default value is "127.0.0.1:8787"
</td>
</tr>
<tr><td class="code">rpc_framework_type</td>
<td> The control path operations can also leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium".
</td>
</tr>
<tr><td class="code">openfam_model</td>
<td> Openfam_model specifies if memory servers are resident in same nodes as clients (shared_memory mode) or distributed across nodes other than client. Default value is "memory_server". Other option is "shared_memory".
</td>
</tr>
<tr><td class="code">resource_release</td>
<td> This option is to enable or disable resource relinquishment in FAM. Default value is "enable". Other option is "disable".
</td>
</tr>
</tbody>
</table>
<p>Sample configuration files are provided in the source under <code>openfam_parent_directory/config</code>.</p>
</article>
</section>
<footer>
<p>Copyright 2021-23, Hewlett Packard Enterprise Development Co, LLP</p>
</footer>
</body>
</html>