Skip to content

Commit e17c57b

Browse files
eff-kayeff-kay
authored
Merge pull request #49 from slashml/update-docs
model deployment
2 parents 3dff527 + 262c16d commit e17c57b

31 files changed

+853
-1002
lines changed

‎docs/api_reference/model_deployment.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
==============
2+
Model Deployment API
3+
==============
4+
5+
.. mdinclude:: snippets/model_deployment_1.md
6+
7+
.. mdinclude:: snippets/model_deployment_2.md
8+
.. mdinclude:: snippets/model_deployment_3.md
9+
.. mdinclude:: snippets/model_deployment_4.md

docs/services/snippets/job_status.md renamed to docs/api_reference/snippets/job_status.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
### Check status of job
22

3-
Now that the audio file has been submitted for transcription, we can make requests to GET the status of the transcription, and eventually the result of the transcription.
3+
The request API is similar to all job submissions. We can make requests to GET the status of the jobs, and eventually the result of the submitted job, i.e. transcription, or speechification.
44

55
```bash
6-
GET https://api.slashml.com/speech-to-text/v1/status/YOUR-JOB-ID/
6+
GET https://api.slashml.com/speech-to-text/v1/jobs/YOUR-JOB-ID/
77
```
88

99
#### Request

‎docs/api_reference/snippets/model_deployment_1.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
This services allows you to fetch the status of a deployed model and call predictions on the exposed API.
2+
3+
## Time to Integrate
4+
5+
Less than 5 minute

‎docs/api_reference/snippets/model_deployment_2.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
## Instructions
2+
3+
1. (Optional) For deployment we recommend using the python SDK. However, you can also push your model using an API. To do this, you will first have to install `truss` to create a truss folder locally. Then compress that truss folder into a `.tar.gz` file. Then send a post `POST` request to `https://api.slashml.com/model-deployment/v1/models` with the compressed file as the body of the request. Save the `id` in the response object.
4+
2. Check the status of the model deployment by sending a `GET` request to `https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/status`
5+
3. You can make predctions on the deployed model by sending a `POST` request to `https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/predict`. The body should contain a json object with `model_input` which is the input prompt to the model.
6+
7+
8+
## Code Blocks
9+
10+
### Submit model for deployment
11+
12+
Install truss
13+
14+
```
15+
pip install truss
16+
```
17+
18+
19+
You can the create a truss object by running the following command from within Python
20+
21+
22+
```python
23+
24+
# you might have to install transfomers and torch
25+
from transformers import pipeline
26+
27+
def train_model():
28+
# Bring in model from huggingface
29+
return pipeline('fill-mask', model='bert-base-uncased')
30+
31+
my_model = train_model()
32+
33+
# save the model
34+
truss.create(my_model, 'my_model')
35+
```
36+
37+
Then convert the folder into a `.tar.gz` file
38+
39+
```bash
40+
tar -czvf my_model.tar.gz my_model
41+
```
42+
43+
#### Request
44+
45+
Then send a post `POST` request to `https://api.slashml.com/model-deployment/v1/models` with the compressed file as the body of the request. Save the `id` in the response object.
46+
47+
```python
48+
import requests
49+
50+
url = "https://api.slashml.com/model-deployment/v1/models/"
51+
52+
payload={'model_name': 'test-dep-model'}
53+
54+
files=[
55+
('model_file',('my_model.tar.gz',open('path/to/my_model.tar.gz','rb'),'application/octet-stream'))
56+
]
57+
58+
headers = {
59+
'Authorization': 'Token YOUR_TOKEN'
60+
}
61+
62+
response = requests.request("POST", url, headers=headers, data=payload, files=files)
63+
64+
print(response.json())
65+
```
66+
67+
68+
#### Response (200)
69+
70+
```bash
71+
{
72+
"id": "a5822206-9680-444c-87ec-4b66a7bcfc26",
73+
"created": "2023-06-13T06:38:55.311751Z",
74+
"status": "IN_PROGRESS",
75+
"name": "'test-dep-model'"
76+
}
77+
```
78+
79+
#### Response (400)
80+
81+
```python
82+
{
83+
"error" : {
84+
"message" : "something bad happened",
85+
}
86+
}
87+
```

‎docs/api_reference/snippets/model_deployment_3.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
### Check status of model
2+
3+
```bash
4+
GET https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/status
5+
```
6+
7+
#### Request
8+
9+
```python
10+
import requests
11+
12+
url = 'https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/status'
13+
14+
headers = {
15+
'Authorization': 'Token <YOUR_API_KEY>'
16+
}
17+
18+
response = requests.get(url, headers=headers, data=payload)
19+
print(response.json())
20+
21+
```
22+
23+
#### Response (200) - MODEL-READy
24+
25+
```bash
26+
{
27+
# keep track of the id for later
28+
"id": "ozfv3zim7-9725-4b54-9b71-f527bc21e5ab",
29+
"created": "2023-06-13T06:38:55.311751Z",
30+
"name": "test-dep-model",
31+
"status": "MODEL_READY",
32+
"name": "test-dep-model",
33+
}
34+
```
35+
36+
#### Response (400) - Error
37+
38+
```bash
39+
{
40+
"error" : {
41+
"message" : "something bad happened"
42+
}
43+
}
44+
45+
```
46+
47+
> Note:
48+
> The status will go from 'QUEUED' to 'BUILDING_MODEL' to 'DEPLOYING_MODEL' to 'MODEL_READY'. If there's an error processing your input, the status will go to 'FAILURE' and there will be an 'ERROR' key in the response JSON which will contain more information.

‎docs/api_reference/snippets/model_deployment_4.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
### Submit a prediction to the model
2+
3+
4+
#### Request
5+
6+
Then send a post `POST` request to `https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/predict` with the model-input as the body of the request.
7+
8+
```python
9+
import requests
10+
import json
11+
12+
url = "https://api.slashml.com/model-deployment/v1/models/YOUR-MODEL-ID/predict"
13+
14+
payload = json.dumps({
15+
"model_input": [
16+
"steve jobs is the [MASK] of apple"
17+
]
18+
})
19+
20+
headers = {
21+
'Authorization': 'Token a7011983a0f3d64ee113317b1e36f8e5bf56c14a',
22+
'Content-Type': 'application/json'
23+
}
24+
25+
response = requests.request("POST", url, headers=headers, data=payload)
26+
27+
print(response.text)
28+
```
29+
30+
31+
#### Response (200)
32+
33+
```bash
34+
{
35+
"id": "a5822206-9680-444c-87ec-4b66a7bcfc26",
36+
"model_input": [
37+
"steve jobs is the [MASK] of apple"
38+
],
39+
"model_response": {
40+
"predictions": [
41+
{
42+
"score": 0.516463041305542,
43+
"sequence": "steve jobs is the founder of apple",
44+
"token": 3910,
45+
"token_str": "founder"
46+
},
47+
{
48+
"score": 0.3604991137981415,
49+
"sequence": "steve jobs is the ceo of apple",
50+
"token": 5766,
51+
"token_str": "ceo"
52+
},
53+
{
54+
"score": 0.04929964989423752,
55+
"sequence": "steve jobs is the president of apple",
56+
"token": 2343,
57+
"token_str": "president"
58+
},
59+
{
60+
"score": 0.021112028509378433,
61+
"sequence": "steve jobs is the creator of apple",
62+
"token": 8543,
63+
"token_str": "creator"
64+
},
65+
{
66+
"score": 0.008550147525966167,
67+
"sequence": "steve jobs is the father of apple",
68+
"token": 2269,
69+
"token_str": "father"
70+
}
71+
]
72+
}
73+
}
74+
```
75+
76+
#### Response (400)
77+
78+
```python
79+
{
80+
"error": "some error occured when requesting job status",
81+
"full_message": [
82+
"{'error': ErrorDetail(string='model not ready', code='permission_denied'), 'reasons': [ErrorDetail(string='model not ready', code='permission_denied'), ErrorDetail(string='model is still being built or deployed', code='permission_denied')]}"
83+
]
84+
}
85+
```

docs/services/snippets/speech_to_text_1.md renamed to docs/api_reference/snippets/speech_to_text_1.md

File renamed without changes.

docs/services/snippets/speech_to_text_2.md renamed to docs/api_reference/snippets/speech_to_text_2.md

File renamed without changes.

‎docs/api_reference/snippets/speech_to_text_3.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
### Convert audio to text
2+
3+
The body of the request should contain a field `uploaded_audio_url`, the value of which shall contain the link to the uploaded audio url, and `service_provider` which is the name of the service provider you want to use.
4+
5+
```bash
6+
POST https://api.slashml.com/speech-to-text/v1/jobs/
7+
```
8+
9+
#### Request
10+
11+
```python
12+
import requests
13+
14+
url = 'https://api.slashml.com/speech-to-text/v1/jobs/'
15+
16+
payload = {
17+
"uploaded_audio_url": "https://cdn.slashml.com/upload/ccbbbfaf-f319-4455-9556-272d48faaf7f",
18+
"service_provider": 'assembly'
19+
}
20+
headers = {
21+
"Authorization": "Token <YOUR_API_KEY>",
22+
}
23+
24+
response = requests.post(url, headers=headers, data=payload)
25+
print(response.text)
26+
```
27+
28+
#### Response (200)
29+
30+
```bash
31+
{
32+
# keep track of the id for later
33+
"id": "ozfv3zim7-9725-4b54-9b71-f527bc21e5ab",
34+
# note that the status is now "processing"
35+
"status": "IN_PROGESS",
36+
"audio_duration": null,
37+
"audio_url": "https://cdn.slashml.com/upload/ccbbbfaf-f319-4455-9556-272d48faaf7f",
38+
"text": null,
39+
}
40+
```
41+
42+
#### Response (400)
43+
44+
```bash
45+
{
46+
"error" : {
47+
"message" : "something bad happened"
48+
}
49+
}
50+
```
51+
52+
> Note:
53+
> The 'id' will be used to fetch the status of the job, in the status endpoint.

docs/services/snippets/text_summarization_1.md renamed to docs/api_reference/snippets/text_summarization_1.md

File renamed without changes.

0 commit comments

Comments
 (0)