fablib¶
This module exports a FablibManager
class and a
fablib
class available, which allows you to, among other
things:
Query FABRIC testbed resources.
Create, modify, and delete slices.
Manage the SSH keys you use with FABRIC.
etc.
In most cases you would need to create a FablibManager
instance to interact with FABRIC testbed:
from fabrictestbed_extensions.fablib.fablib import FablibManager
fablib = FablibManager()
slice = fablib.new_slice(name="MySlice")
node = slice.add_node(name="node1")
slice.submit();
See FABRIC project’s Jupyter notebook examples for more complete code samples.
Note
Some configuration in the form of a configuration file, environment
variables, or FablibManager
constructor parameters is
required for the library to work. Please see the FABRIC project’s
documentation on getting started.
- class fabrictestbed_extensions.fablib.fablib.fablib¶
Convenience static methods to work with FABRIC testbed.
- static delete_all(progress: bool = True)¶
Deletes all slices on the slice manager.
- Parameters:
progress (Bool) – optional progress printing to stdout
- static delete_slice(slice_name: str | None = None)¶
Deletes a slice by name.
- Parameters:
slice_name (String) – the name of the slice to delete
- static get_available_resources(update: bool = False) Resources ¶
Get the available resources.
Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.
- Parameters:
update (Bool) – update
- Returns:
Available Resources object
- Return type:
- static get_bastion_host() str ¶
Gets the FABRIC Bastion host address.
- Returns:
Bastion host public address
- Return type:
String
- static get_bastion_key_filename() str ¶
Gets the FABRIC Bastion key filename.
- Returns:
FABRIC Bastion key filename
- Return type:
String
- static get_bastion_username() str ¶
Gets the FABRIC Bastion username.
- Returns:
FABRIC Bastion username
- Return type:
String
- static get_config() Dict[str, str] ¶
Gets a dictionary mapping keywords to configured FABRIC environment variable values values.
- Returns:
dictionary mapping keywords to FABRIC values
- Return type:
Dict[String, String]
- static get_credmgr_host() str ¶
Gets the credential manager host site value.
- Returns:
the credential manager host site
- Return type:
String
- static get_default_slice_key() Dict[str, str] ¶
Gets the current default_slice_keys as a dictionary containing the public and private slice keys.
Important! Slice key management is underdevelopment and this functionality will likely change going forward.
- Returns:
default_slice_key dictionary from superclass
- Return type:
Dict[String, String]
- static get_default_slice_private_key_file() str ¶
Gets the path to the default slice private key file.
Important! Slices key management is underdevelopment and this functionality will likely change going forward.
- Returns:
the path to the slice private key on this fablib object
- Return type:
String
- static get_default_slice_private_key_passphrase() str ¶
Gets the passphrase to the default slice private key.
Important! Slices key management is underdevelopment and this functionality will likely change going forward.
- Returns:
the passphrase to the slice private key on this fablib object
- Return type:
String
- static get_default_slice_public_key() str ¶
Gets the default slice public key.
Important! Slice key management is underdevelopment and this functionality will likely change going forward.
- Returns:
the slice public key on this fablib object
- Return type:
String
- static get_default_slice_public_key_file() str ¶
Gets the path to the default slice public key file.
Important! Slice key management is underdevelopment and this functionality will likely change going forward.
- Returns:
the path to the slice public key on this fablib object
- Return type:
String
- static get_fabric_token() str ¶
Gets the FABRIC token location.
- Returns:
FABRIC token location
- Return type:
String
- static get_facility_ports() FacilityPorts ¶
Get a string used to print a tabular list of facility ports
- Returns:
tabulated string of facility ports
- Return type:
str
- static get_fim_slice(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) List[OrchestratorSlice] ¶
Not intended for API use.
Gets a list of fim slices from the slice manager.
By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.
- Parameters:
excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
a list of slices
- Return type:
List[Slice]
- static get_image_names() List[str] ¶
Gets a list of available image names.
- Returns:
list of image names as strings
- Return type:
list[str]
- static get_links() Links ¶
Get a string used to print a tabular list of links
- Returns:
tabulated string of links
- Return type:
str
- static get_log_level()¶
Gets the current log level for logging
- static get_orchestrator_host() str ¶
Gets the orchestrator host site value.
- Returns:
the orchestrator host site
- Return type:
String
- static get_random_site(avoid: List[str] = []) str ¶
Get a random site.
- Parameters:
avoid – list of site names to avoid choosing
- Returns:
one site name
- Return type:
String
- static get_random_sites(count: int = 1, avoid: List[str] = []) List[str] ¶
Get a list of random sites names. Each site will be included at most once.
- Parameters:
count (int) – number of sites to return.
avoid (List[String]) – list of site names to avoid choosing
- Returns:
list of random site names.
- Return type:
List[Sting]
- static get_resources() Resources ¶
Get a reference to the resources object. The resources object is used to query for available resources and capacities.
- Returns:
the resources object
- Return type:
- static get_site_advertisement(site: str) Node ¶
Not intended for API use.
Given a site name, gets fim topology object for this site.
- Parameters:
site (String) – a site name
- Returns:
fim object for this site
- Return type:
- static get_site_names() List[str] ¶
Gets a list of all available site names.
- Returns:
list of site names as strings
- Return type:
list[str]
- static get_slice(name: str | None = None, slice_id: str | None = None, user_only: bool = True) Slice ¶
Gets a slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.
If both a name and slice_id are provided, the slice matching the slice_id will be returned.
- Parameters:
name (String) – The name of the desired slice
slice_id (String) – The ID of the desired slice
user_only (bool) – True indicates return own slices; False indicates return project slices
- Raises:
Exception: if slice name or slice id are not inputted
- Returns:
the slice, if found
- Return type:
- static get_slice_manager() SliceManager ¶
Not intended as API call
Gets the slice manager of this fablib object.
- Returns:
the slice manager on this fablib object
- Return type:
SliceManager
- static get_slices(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) List[Slice] ¶
Gets a list of slices from the slice manager.
By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.
- Parameters:
excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
a list of slices
- Return type:
List[Slice]
- static init_fablib()¶
Not intended to be called by the user.
Static initializer for the fablib object.
- static list_facility_ports() object ¶
Print the facility ports in pretty format
- Returns:
Formatted list of facility ports
- Return type:
object
- static list_links() object ¶
Print the links in pretty format
- Returns:
Formatted list of links
- Return type:
object
- static list_sites(latlon: bool = True) object ¶
Get a string used to print a tabular list of sites with state
- Returns:
tabulated string of site state
- Return type:
str
- static new_slice(name: str) Slice ¶
Creates a new slice with the given name.
- Parameters:
name (String) – the name to give the slice
- Returns:
a new slice
- Return type:
- static set_log_level(log_level: str)¶
Sets the current log level for logging
- Options: logging.DEBUG
logging.INFO logging.WARNING logging.ERROR logging.CRITICAL
- Parameters:
log_level (Level) – new log level
- static show_site(site_name: str)¶
Get a string used to print tabular info about a site
- Parameters:
site_name (String) – the name of a site
- Returns:
tabulated string of site state
- Return type:
String
- class fabrictestbed_extensions.fablib.fablib.FablibManager(fabric_rc: str | None = None, credmgr_host: str | None = None, orchestrator_host: str | None = None, core_api_host: str | None = None, token_location: str | None = None, project_id: str | None = None, bastion_username: str | None = None, bastion_key_location: str | None = None, log_level: str = 'INFO', log_file: str = '/tmp/fablib/fablib.log', data_dir: str = '/tmp/fablib', output: str | None = None, execute_thread_pool_size: int = 64, offline: bool = False, auto_token_refresh: bool = True, **kwargs)¶
FablibManager
is the main interface to FABRIC services.A
FablibManager
object is used to query FABRIC testbed for available resources, create and configure slices, manage SSH keys in nodes in slices and FABRIC’s bastion host, etc. This requires some configuration, which is gathered from:constructor parameters (high priority)
a configuration file (medium priority)
environment variables (low priority)
defaults (if needed, and when possible)
Typically you would use the configuration file located at
"${HOME}/work/fabric_config/fabric_rc"
, and/or environment variables.- Parameters:
fabric_rc – Path to fablib configuration file. Defaults to
"${HOME}/work/fabric_config/fabric_rc"
.credmgr_host – Name of credential manager host.
orchestrator_host – Name of FABRIC orchestrator host.
fabric_token – Path to the file that contains your FABRIC auth token.
project_id – Your FABRIC project ID, obtained from https://cm.fabric-testbed.net/, usually via FABRIC portal.
bastion_username – Your username on FABRIC bastion host, obtained from FABRIC portal.
bastion_key_filename – Path to your bastion SSH key.
log_file – Path where fablib logs are written; defaults to
"/tmp/fablib/fablib.log"
.log_level – Level of detail in the logs written. Defaults to
"DEBUG"
; other possible log levels are"INFO"
,"WARNING"
,"ERROR"
, and"CRITICAL"
, in reducing order of verbosity.data_dir – directory for fablib to store temporary data.
output – Format of fablib output; can be either
"pandas"
or"text"
. Defaults to"pandas"
in a Jupyter notebook environment;"text"
otherwise.execute_thread_pool_size – Number of worker threads in the thread pool fablib uses to execute commands in nodes. Defaults to 64.
offline – Avoid using FABRIC services when initializing. This is
False
by default, and set toTrue
only in some unit tests.auto_token_refresh – Auto refresh tokens
- create_bastion_keys(*, bastion_key_location: str | None = None, store_pubkey: bool = True, overwrite: bool = False)¶
Create Bastion Keys
- Parameters:
bastion_key_location (str) – bastion key location
store_pubkey (bool) – flag indicating if the public key should be saved
overwrite (bool) – overwrite the bastion key file if it exists already
- create_sliver_keys(*, sliver_priv_key_location: str | None = None, store_pubkey: bool = True, overwrite: bool = False)¶
Create Sliver Keys
- Parameters:
sliver_priv_key_location (str) – sliver key location
store_pubkey (bool) – flag indicating if the public key should be saved
overwrite (bool) – overwrite the bastion key file if it exists already
- create_ssh_config(overwrite: bool = False)¶
Create SSH config file
- Parameters:
overwrite (bool) – overwrite the configuration if True, return otherwise
- delete_all(progress: bool = True)¶
Deletes all slices on the slice manager.
- Parameters:
progress (Bool) – optional progress printing to stdout
- delete_slice(slice_name: str | None = None)¶
Deletes a slice by name.
- Parameters:
slice_name (String) – the name of the slice to delete
- determine_bastion_username()¶
Determine Bastion Username. Query User Information from Core API and updates the bastion username
- get_available_resources(update: bool = False, force_refresh: bool = False) Resources ¶
Get the available resources.
Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.
- Parameters:
update
force_refresh
- Returns:
Available Resources object
- get_facility_ports(update: bool = True) FacilityPorts ¶
Get the facility ports.
Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.
- Parameters:
update
- Returns:
Links
- get_fim_slices(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) List[OrchestratorSlice] ¶
Gets a list of fim slices from the slice manager.
This is not recommended for most users and should only be used to bypass fablib inorder to create custom low-level functionality.
By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.
- Parameters:
excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
a list of fim models of slices
- Return type:
List[Slice]
- get_links(update: bool = True) Links ¶
Get the links.
Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.
- Parameters:
update
- Returns:
Links
- get_random_site(avoid: List[str] = [], filter_function=None, update: bool = True) str ¶
Get a random site.
- Parameters:
avoid (List[String]) – list of site names to avoid choosing
filter_function – filter_function
update (bool) – flag indicating if fetch latest availability information
- Returns:
one site name
- Return type:
String
- get_random_sites(count: int = 1, avoid: List[str] = [], filter_function=None, update: bool = True, unique: bool = True) List[str] ¶
Get a list of random sites names. Each site will be included at most once.
- Parameters:
count (int) – number of sites to return.
avoid (List[String]) – list of site names to avoid chosing
filter_function – filter_function
update (bool) – flag indicating if fetch latest availability information
unique
- Returns:
one site name
- Returns:
list of random site names.
- Return type:
List[Sting]
- get_resources(update: bool = True, force_refresh: bool = False) Resources ¶
Get a reference to the resources object. The resources object is used to query for available resources and capacities.
- Returns:
the resources object
- Return type:
- get_site_advertisement(site: str) Node ¶
Not intended for API use.
Given a site name, gets fim topology object for this site.
- Parameters:
site (String) – a site name
- Returns:
fim object for this site
- Return type:
- get_site_names() List[str] ¶
Gets a list of all available site names.
- Returns:
list of site names as strings
- Return type:
list[str]
- get_slice(name: str | None = None, slice_id: str | None = None, user_only: bool = True) Slice ¶
Gets a slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.
If both a name and slice_id are provided, the slice matching the slice_id will be returned.
- Parameters:
name (String) – The name of the desired slice
slice_id (String) – The ID of the desired slice
user_only (bool) – True indicates return own slices; False indicates return project slices
- Raises:
Exception: if slice name or slice id are not inputted
- Returns:
the slice, if found
- Return type:
- get_slice_manager() SliceManager ¶
Not intended as API call
Gets the slice manager of this fablib object.
- Returns:
the slice manager on this fablib object
- Return type:
SliceManager
- get_slices(excludes: List[SliceState] = [Dead, Closing], slice_name: str | None = None, slice_id: str | None = None, user_only: bool = True) List[Slice] ¶
Gets a list of slices from the slice manager.
By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.
- Parameters:
excludes (List[SliceState]) – A list of slice states to exclude from the output list
slice_name
slice_id
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
a list of slices
- Return type:
List[Slice]
- get_user_info() dict ¶
Get User information
:return returns a dictionary containing User’s Information :rtype: dict
- static is_jupyter_notebook() bool ¶
Test for running inside a jupyter notebook
- Returns:
bool, True if in jupyter notebook
- Return type:
bool
- list_facility_ports(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names=True) object ¶
Lists all the facility ports and their attributes.
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields/columns.
Example: TODO
filter_function: A lambda function to filter data by field values.
Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10
- Parameters:
output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)
- Returns:
table in format specified by output parameter
- Return type:
Object
- list_links(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names=True) object ¶
Lists all the links and their attributes.
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields/columns.
Example: TODO
filter_function: A lambda function to filter data by field values.
Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10
- Parameters:
output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)
- Returns:
table in format specified by output parameter
- Return type:
Object
- list_sites(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names: bool = True, force_refresh: bool = False, latlon: bool = True) object ¶
Lists all the sites and their attributes.
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields/columns.
Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]
filter_function: A lambda function to filter data by field values.
Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10
- Parameters:
output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)
force_refresh (bool)
latlon – convert address to latlon, makes online call to openstreetmaps.org
- Returns:
table in format specified by output parameter
- Return type:
Object
- list_slices(excludes=[Dead, Closing], output=None, fields=None, quiet=False, filter_function=None, pretty_names=True, user_only: bool = True)¶
Lists all the slices created by a user.
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields/columns.
Example: fields=[‘Name’,’State’]
filter_function: A lambda function to filter data by field values.
Example: filter_function=lambda s: s[‘State’] == ‘Configuring’
- Parameters:
excludes (list[slice.state]) – slice status to exclude
output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
pretty_names (bool) – pretty_names
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
table in format specified by output parameter
- Return type:
Object
- new_slice(name: str) Slice ¶
Creates a new slice with the given name.
- Parameters:
name (String) – the name to give the slice
- Returns:
a new slice
- Return type:
- probe_bastion_host() bool ¶
See if bastion will admit us with our configuration.
Bastion hosts are configured to block hosts that attempts to use it with too many repeated authentication failures. We want to avoid that.
Returns
True
if connection attempt succeeds. Raises an error in the event of failure.
- set_slice_manager(slice_manager: SliceManager)¶
Not intended as API call
Sets the slice manager of this fablib object.
- Parameters:
slice_manager (SliceManager) – the slice manager to set
- show_config(output: str | None = None, fields: list[str] | None = None, quiet: bool = False, pretty_names=True)¶
Show a table containing the current FABlib configuration parameters.
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields.
Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]
- Parameters:
output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool)
- Returns:
table in format specified by output parameter
- Return type:
Object
- show_site(site_name: str, output: str | None = None, fields: list[str] | None = None, quiet: bool = False, pretty_names=True, latlon=True)¶
Show a table with all the properties of a specific site
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields.
Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]
- Parameters:
site_name (str) – the name of a site
output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool)
latlon (bool) – convert address to lat/lon
- Returns:
table in format specified by output parameter
- Return type:
Object
- show_slice(name: str | None = None, id: str | None = None, output=None, fields=None, quiet=False, pretty_names=True, user_only: bool = True)¶
Show a table with all the properties of a specific site
There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.
- output: ‘text’: string formatted with tabular
‘pandas’: pandas dataframe ‘json’: string in json format
fields: json output will include all available fields.
Example: fields=[‘Name’,’State’]
- Parameters:
name (str) – the name of a slice
id – the slice id
output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool) – pretty_names
user_only (bool) – True indicates return own slices; False indicates return project slices
- Returns:
table in format specified by output parameter
- Return type:
Object
- validate_and_update_bastion_keys()¶
Validate Bastion Key; if key does not exist or is expired, it create bastion keys
- validate_config()¶
Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully
Checks Credential Manager Host is configured properly
Checks Orchestrator Host is configured properly
Checks Core API Host is configured properly
Checks Bastion Host is configured properly
Check Sliver keys exist; create sliver keys if they do not exist
Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist
Check Bastion Username is configured
Check Project Id is configured
Deprecated since version 1.6.5: Use verify_and_configure() instead.
@raises Exception if the configuration is invalid
- verify_and_configure()¶
Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully
Checks Credential Manager Host is configured properly
Checks Orchestrator Host is configured properly
Checks Core API Host is configured properly
Checks Bastion Host is configured properly
Check Sliver keys exist; create sliver keys if they do not exist
Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist
Check Bastion Username is configured
Check Project Id is configured
@raises Exception if the configuration is invalid