From 2cff0d89e4ba6286a6c94b9c6e50b53657d484cd Mon Sep 17 00:00:00 2001 From: saileshwar-skyflow Date: Wed, 5 Feb 2025 20:14:07 +0530 Subject: [PATCH 1/4] SK-1863: Added migration from v1 to v2 guide to readme --- README.md | 245 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 245 insertions(+) diff --git a/README.md b/README.md index bfdab7a3..8f48ce1c 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ This Python SDK is designed to help developers easily implement Skyflow into the - [Requirements](#requirements) - [Configuration](#configuration) - [Service Account Bearer Token Generation](#service-account-bearer-token-generation) + [Migration from v1 to v2](#migrate-from-v1-to-v2) - [Vault APIs](#vault-apis) - [Insert data into the vault](#insert-data-into-the-vault) - [Detokenize](#detokenize) @@ -328,6 +329,250 @@ try: except SkyflowError as e: print(e) ``` +## Migrate from V1 to V2 + +Below are the steps to migrate the Python SDK from V1 to V2. + +### 1. Authentication Options + +In V2, we have introduced multiple authentication options. +You can now provide credentials in the following ways: + +1. **API Key (Recommended)** +2. **Environment Variables** (SKYFLOW_CREDENTIALS) (Recommended) +3. **Path to your credentials JSON file** +4. **Stringified JSON of your credentials** +5. **Bearer token** + +These options allow you to choose the authentication method that best suits your use case. + +#### V1 (Old): +Passing the token provider function below as a parameter to the Configuration. + +```python +# User defined function to provide access token to the vault apis +def token_provider(): + global bearerToken + if !(is_expired(bearerToken)): + return bearerToken + bearerToken, _ = generate_bearer_token('') + return bearerToken +``` + +#### V2 (New): +Passing one of the following: + +```python +# Option 1: API Key (Recommended) +credentials = { + 'api_key': '', # API key +} + +# Option 2: Environment Variables (Recommended) +# Set SKYFLOW_CREDENTIALS in your environment + +# Option 3: Credentials File +credentials = { + 'path': '', # Path to credentials file +} + +# Option 4: Stringified JSON +credentials = { + 'credentials_string': '', # Credentials as string +} + +# Option 5: Bearer Token +credentials = { + 'token': '', # Bearer token +} +``` + +**Notes:** +1. Use only ONE authentication method. +2. Environment variables take precedence over programmatic configuration. +3. API Key or Environment Variables are recommended for production use. +4. Secure storage of credentials is essential. +5. For overriding behavior and priority order of credentials, please refer to the README. + +### 2. Client Initialization + +In V2, we have introduced a Builder design pattern for client initialization and added support for multi-vault. This allows you to configure multiple vaults during client initialization. + +In V2, the log level is tied to each individual client instance. + +During client initialization, you can pass the following parameters: + +1. **vault_id** and **cluster_id**: These values are derived from the vault ID & vault URL. +2. **env**: Specify the environment (e.g., SANDBOX or PROD). +3. **credentials**: The necessary authentication credentials. + +#### V1 (Old): + +```python +# Initializing a Skyflow Client instance with a SkyflowConfiguration object +config = Configuration('', '', token_provider) +client = Client(config) +``` + +#### V2 (New): + +```python +# Initializing a Skyflow Client instance +client = ( + Skyflow.builder() + .add_vault_config({ + 'vault_id': 'VAULT_ID', # Primary vault + 'cluster_id': 'CLUSTER_ID', # ID from your vault URL e.g., https://{clusterId}.vault.skyflowapis.com + 'env': Env.PROD, # Env by default it is set to PROD + 'credentials': credentials # Individual credentials + }) + .add_skyflow_credentials(credentials) # Skyflow credentials will be used if no individual credentials are passed + .set_log_level(LogLevel.INFO) # set log level by default it is set to ERROR + .build() +) +``` + +**Key Changes:** +1. `vault_url` replaced with `clusterId`. +2. Added environment specification (`env`). +3. Instance-specific log levels. + +### 3. Request & Response Structure + +In V2, with the introduction of constructor parameters, you can now pass parameters to `InsertRequest`. This request need +- **table_name**: The name of the table. +- **values**: An array of objects containing the data to be inserted. +The response will be of type `InsertResponse` class, which contains `inserted_fields` and errors. + +#### V1 (Old): Request Building + +```python +client.insert( + { + "records": [ + { + "table": "cards", + "fields": { + "cardNumber": "41111111111", + "cvv": "123", + }, + } + ] + }, + InsertOptions(True), +) +``` + +#### V2 (New): Request Building + +```python +# Prepare Insertion Data +insert_data = [ + { + 'card_number': '', + 'cvv': '', + }, +] + +table_name = '' # Replace with your actual table name + +# Create Insert Request +insert_request = InsertRequest( + table_name=table_name, + values=insert_data, + return_tokens=True, # Optional: Get tokens for inserted data + continue_on_error=True # Optional: Continue on partial errors +) + +# Perform Secure Insertion +response = skyflow_client.vault(primary_vault_config.get('vault_id')).insert(insert_request) +``` + +#### V1 (Old): Response Structure + +```json +{ + "records": [ + { + "table": "cards", + "fields": { + "cardNumber": "f3907186-e7e2-466f-91e5-48e12c2bcbc1", + "cvv": "1989cb56-63da-4482-a2df-1f74cd0dd1a5", + "skyflow_id": "d863633c-8c75-44fc-b2ed-2b58162d1117" + }, + "request_index": 0 + } + ] +} +``` + +#### V2 (New): Response Structure + +```python +InsertResponse( + inserted_fields=[ + { + 'skyflow_id': 'a8f3ed5d-55eb-4f32-bf7e-2dbf4b9d9097', + 'card_number': '5479-4229-4622-1393' + } + ], + errors=[] +) +``` + +### 4. Request Options + +In V2, we have introduced constructor parameters, allowing you to set options as key-value pairs as parameters in request. + +#### V1 (Old): + +```python +options = InsertOptions( + tokens = True +) +``` + +#### V2 (New): + +```python +insert_request = InsertRequest( + table_name=table_name, + values=insert_data, + return_tokens=True, # Optional: Get tokens for inserted data + continue_on_error=True # Optional: Continue on partial errors +) +``` + +### 5. Request Options + +In V2, we have enriched the error details to provide better debugging capabilities. +The error response now includes: +- **http_status**: The HTTP status code. +- **grpc_code**: The gRPC code associated with the error. +- **details & message**: A detailed description of the error. +- **request_id**: A unique request identifier for easier debugging. + +#### V1 (Old) Error Structure: + +```json +{ + "code": "", + "message": "" +} +``` + +#### V2 (New) Error Structure: + +```json +{ + "http_status": "", + "grpc_code": "", + "http_code": "", + "message": "", + "request_id": "", + "details": [ "
" ] +} +``` ## Vault APIs From e1c79a5c9dcaec8ba020f053c973c4af29fc60d4 Mon Sep 17 00:00:00 2001 From: saileshwar-skyflow Date: Thu, 6 Feb 2025 17:49:57 +0530 Subject: [PATCH 2/4] SK-1863: Refactored README --- README.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 8f48ce1c..21de6756 100644 --- a/README.md +++ b/README.md @@ -339,7 +339,7 @@ In V2, we have introduced multiple authentication options. You can now provide credentials in the following ways: 1. **API Key (Recommended)** -2. **Environment Variables** (SKYFLOW_CREDENTIALS) (Recommended) +2. **Environment Variables** (SKYFLOW_CREDENTIALS) (**Recommended**) 3. **Path to your credentials JSON file** 4. **Stringified JSON of your credentials** 5. **Bearer token** @@ -365,7 +365,7 @@ Passing one of the following: ```python # Option 1: API Key (Recommended) credentials = { - 'api_key': '', # API key + 'api_key': '', # API key } # Option 2: Environment Variables (Recommended) @@ -373,26 +373,26 @@ credentials = { # Option 3: Credentials File credentials = { - 'path': '', # Path to credentials file + 'path': '', # Path to credentials file } # Option 4: Stringified JSON credentials = { - 'credentials_string': '', # Credentials as string + 'credentials_string': '', # Credentials as string } # Option 5: Bearer Token credentials = { - 'token': '', # Bearer token + 'token': '', # Bearer token } ``` **Notes:** -1. Use only ONE authentication method. -2. Environment variables take precedence over programmatic configuration. -3. API Key or Environment Variables are recommended for production use. -4. Secure storage of credentials is essential. -5. For overriding behavior and priority order of credentials, please refer to the README. +- Use only ONE authentication method. +- Environment variables take precedence over programmatic configuration. +- API Key or Environment Variables are recommended for production use. +- Secure storage of credentials is essential. +- For overriding behavior and priority order of credentials, please refer to the README. ### 2. Client Initialization @@ -402,9 +402,9 @@ In V2, the log level is tied to each individual client instance. During client initialization, you can pass the following parameters: -1. **vault_id** and **cluster_id**: These values are derived from the vault ID & vault URL. -2. **env**: Specify the environment (e.g., SANDBOX or PROD). -3. **credentials**: The necessary authentication credentials. +- **`vault_id`** and **`cluster_id`**: These values are derived from the vault ID & vault URL. +- **`env`**: Specify the environment (e.g., SANDBOX or PROD). +- **`credentials`**: The necessary authentication credentials. #### V1 (Old): @@ -421,8 +421,8 @@ client = Client(config) client = ( Skyflow.builder() .add_vault_config({ - 'vault_id': 'VAULT_ID', # Primary vault - 'cluster_id': 'CLUSTER_ID', # ID from your vault URL e.g., https://{clusterId}.vault.skyflowapis.com + 'vault_id': '', # Primary vault + 'cluster_id': '', # ID from your vault URL e.g., https://{clusterId}.vault.skyflowapis.com 'env': Env.PROD, # Env by default it is set to PROD 'credentials': credentials # Individual credentials }) @@ -433,15 +433,15 @@ client = ( ``` **Key Changes:** -1. `vault_url` replaced with `clusterId`. -2. Added environment specification (`env`). -3. Instance-specific log levels. +- `vault_url` replaced with `cluster_Id`. +- Added environment specification (`env`). +- Instance-specific log levels. ### 3. Request & Response Structure In V2, with the introduction of constructor parameters, you can now pass parameters to `InsertRequest`. This request need -- **table_name**: The name of the table. -- **values**: An array of objects containing the data to be inserted. +- **`table_name`**: The name of the table. +- **`values`**: An array of objects containing the data to be inserted. The response will be of type `InsertResponse` class, which contains `inserted_fields` and errors. #### V1 (Old): Request Building @@ -485,7 +485,7 @@ insert_request = InsertRequest( ) # Perform Secure Insertion -response = skyflow_client.vault(primary_vault_config.get('vault_id')).insert(insert_request) +response = skyflow_client.vault(primary_vault_config.get('')).insert(insert_request) ``` #### V1 (Old): Response Structure From ed4a6739aca67d4f62a0186e05ed1bae7d7eb728 Mon Sep 17 00:00:00 2001 From: saileshwar-skyflow Date: Thu, 6 Feb 2025 19:42:24 +0530 Subject: [PATCH 3/4] SK-1863: Refactored README --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 21de6756..eb5593ec 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This Python SDK is designed to help developers easily implement Skyflow into the - [Requirements](#requirements) - [Configuration](#configuration) - [Service Account Bearer Token Generation](#service-account-bearer-token-generation) - [Migration from v1 to v2](#migrate-from-v1-to-v2) + - [Migration from v1 to v2](#migrate-from-v1-to-v2) - [Vault APIs](#vault-apis) - [Insert data into the vault](#insert-data-into-the-vault) - [Detokenize](#detokenize) @@ -338,11 +338,11 @@ Below are the steps to migrate the Python SDK from V1 to V2. In V2, we have introduced multiple authentication options. You can now provide credentials in the following ways: -1. **API Key (Recommended)** -2. **Environment Variables** (SKYFLOW_CREDENTIALS) (**Recommended**) -3. **Path to your credentials JSON file** -4. **Stringified JSON of your credentials** -5. **Bearer token** +- **API Key (Recommended)** +- **Environment Variable** (`SKYFLOW_CREDENTIALS`) (**Recommended**) +- **Path to your credentials JSON file** +- **Stringified JSON of your credentials** +- **Bearer token** These options allow you to choose the authentication method that best suits your use case. From 53d0900c5e0b4db7963a740ee9ee53a2bc921b64 Mon Sep 17 00:00:00 2001 From: saileshwar-skyflow Date: Fri, 7 Feb 2025 12:51:25 +0530 Subject: [PATCH 4/4] SK-1863: Updated notes for auth options --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index eb5593ec..f3d07545 100644 --- a/README.md +++ b/README.md @@ -389,7 +389,6 @@ credentials = { **Notes:** - Use only ONE authentication method. -- Environment variables take precedence over programmatic configuration. - API Key or Environment Variables are recommended for production use. - Secure storage of credentials is essential. - For overriding behavior and priority order of credentials, please refer to the README.