| --- |
| language: |
| - en |
| license: apache-2.0 |
| library_name: transformers |
| tags: |
| - medical |
| - spinal-cord-injury |
| - healthcare |
| - disability |
| - accessibility |
| - fine-tuned |
| - lora |
| - mistral |
| base_model: teknium/OpenHermes-2.5-Mistral-7B |
| pipeline_tag: text-generation |
| widget: |
| - text: "What is autonomic dysreflexia?" |
| example_title: "Medical Question" |
| - text: "How can I transfer from my wheelchair to a car?" |
| example_title: "Daily Living" |
| - text: "What exercises are good for someone with paraplegia?" |
| example_title: "Exercise & Rehabilitation" |
| model-index: |
| - name: sci-assistant |
| results: [] |
| --- |
| |
| # SCI Assistant - Spinal Cord Injury Specialized AI Assistant |
|
|
| A specialized AI assistant fine-tuned specifically for people with spinal cord injuries (SCI). This model is based on OpenHermes-2.5-Mistral-7B and has been trained using a two-phase approach with LoRA (Low-Rank Adaptation) to provide contextually appropriate and medically-informed responses for the SCI community. |
|
|
| ## Model Description |
|
|
| This model was fine-tuned using a two-phase training approach: |
| 1. **Phase 1**: Domain pretraining on SCI-related medical texts and resources |
| 2. **Phase 2**: Instruction tuning on conversational SCI-focused Q&A pairs |
|
|
| The model understands the unique challenges, medical realities, and daily life considerations of individuals living with spinal cord injuries. |
|
|
| ## Training Details |
|
|
| - **Base Model**: teknium/OpenHermes-2.5-Mistral-7B |
| - **Training Method**: QLoRA (4-bit quantization with LoRA adapters) |
| - **Training Data**: 119,117 total entries (35,779 domain text + 83,337 instruction pairs) |
| - **Hardware**: RTX 4070 Super (12GB VRAM) |
| - **Training Time**: ~20 hours total (Phase 1 + Phase 2) |
|
|
| ## Usage |
|
|
| This repository contains both the LoRA adapter and the full merged model. Choose the option that works best for you: |
|
|
| ### Option 1: Use the Full Merged Model (Recommended) |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("basiphobe/sci-assistant") |
| tokenizer = AutoTokenizer.from_pretrained("basiphobe/sci-assistant") |
| |
| # Example usage |
| prompt = "What are the signs of autonomic dysreflexia?" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_length=200) |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| ``` |
|
|
| ### Option 2: Use the LoRA Adapter (Smaller Download) |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig |
| from peft import PeftModel |
| import torch |
| |
| # Load model |
| bnb_config = BitsAndBytesConfig( |
| load_in_4bit=True, |
| bnb_4bit_compute_dtype=torch.float16, |
| ) |
| |
| base_model = AutoModelForCausalLM.from_pretrained( |
| "teknium/OpenHermes-2.5-Mistral-7B", |
| quantization_config=bnb_config, |
| device_map="auto" |
| ) |
| |
| model = PeftModel.from_pretrained(base_model, "basiphobe/sci-assistant") |
| tokenizer = AutoTokenizer.from_pretrained("basiphobe/sci-assistant") |
| |
| # Format prompt with SCI context |
| system_context = "You are a specialized medical assistant for people with spinal cord injuries. Your responses should always consider the unique needs, challenges, and medical realities of individuals living with SCI." |
| |
| prompt = f"{system_context}\n\n### Instruction:\n{your_question}\n\n### Response:\n" |
| |
| # Generate response |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7) |
| response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) |
| ``` |
|
|
| ## Files in this Repository |
|
|
| - **Full Merged Model**: Ready-to-use model files (`model-*.safetensors`, `config.json`, etc.) |
| - **LoRA Adapter**: Smaller adapter files (`adapter_model.safetensors`, `adapter_config.json`) |
| - **Tokenizer**: Shared tokenizer files for both options |
|
|
| ## GGUF Format Models |
|
|
| This repository also includes GGUF format models optimized for use with **llama.cpp**, **Ollama**, and other GGUF-compatible inference engines. These formats offer excellent performance and compatibility across different platforms. |
|
|
| ### Available GGUF Models |
|
|
| | File | Size | Format | Use Case | RAM Required | |
| |------|------|--------|----------|--------------| |
| | `merged-sci-model.gguf` | 14GB | F16 | Maximum quality inference | ~16GB | |
| | `merged-sci-model-q6_k.gguf` | 5.6GB | Q6_K | High quality with good compression | ~8GB | |
| | `merged-sci-model-q5_k_m.gguf` | 4.8GB | Q5_K_M | Excellent quality/size balance | ~7GB | |
| | `merged-sci-model-q5_k_s.gguf` | 4.7GB | Q5_K_S | Good quality, slightly smaller | ~7GB | |
| | `merged-sci-model-q4_k_m.gguf` | 4.1GB | Q4_K_M | Balanced quality/performance | ~6GB | |
| |
| ### Usage with Ollama |
| |
| **1. Download and create Modelfile:** |
| ```bash |
| # Download the Q5_K_M model (recommended balance of quality/size) |
| wget https://huggingface.co/basiphobe/sci-assistant/resolve/main/merged-sci-model-q5_k_m.gguf |
| |
| # Create Modelfile |
| cat > Modelfile << 'EOF' |
| FROM ./merged-sci-model-q5_k_m.gguf |
| TEMPLATE """<|im_start|>system |
| You are a specialized medical assistant for people with spinal cord injuries. Your responses should always consider the unique needs, challenges, and medical realities of individuals living with SCI.<|im_end|> |
| <|im_start|>user |
| {{ .Prompt }}<|im_end|> |
| <|im_start|>assistant |
| """ |
| PARAMETER stop "<|im_start|>" |
| PARAMETER stop "<|im_end|>" |
| PARAMETER temperature 0.7 |
| PARAMETER top_p 0.9 |
| EOF |
| ``` |
| |
| **2. Create and run the model:** |
| ```bash |
| ollama create sci-assistant -f Modelfile |
| ollama run sci-assistant "What are the signs of autonomic dysreflexia?" |
| ``` |
| |
| ### Usage with llama.cpp |
| |
| **1. Install and setup:** |
| ```bash |
| # Clone and build llama.cpp |
| git clone https://github.com/ggerganov/llama.cpp |
| cd llama.cpp |
| make |
| |
| # Download model |
| wget https://huggingface.co/basiphobe/sci-assistant/resolve/main/merged-sci-model-q5_k_m.gguf |
| ``` |
| |
| **2. Interactive chat:** |
| ```bash |
| ./main -m merged-sci-model-q5_k_m.gguf \ |
| --temp 0.7 \ |
| --repeat_penalty 1.1 \ |
| -c 4096 \ |
| --interactive \ |
| --in-prefix "<|im_start|>user\n" \ |
| --in-suffix "<|im_end|>\n<|im_start|>assistant\n" |
| ``` |
| |
| **3. Single prompt:** |
| ```bash |
| ./main -m merged-sci-model-q5_k_m.gguf \ |
| --temp 0.7 \ |
| -c 2048 \ |
| -p "<|im_start|>system\nYou are a specialized medical assistant for people with spinal cord injuries.<|im_end|>\n<|im_start|>user\nWhat exercises are good for someone with paraplegia?<|im_end|>\n<|im_start|>assistant\n" |
| ``` |
| |
| ### Performance Comparison |
| |
| - **F16 Model** (`merged-sci-model.gguf`): Maximum quality, largest memory footprint |
| - **Q6_K Model** (`merged-sci-model-q6_k.gguf`): Near-maximum quality with 60% size reduction |
| - **Q5_K_M Model** (`merged-sci-model-q5_k_m.gguf`): Excellent quality retention, good balance |
| - **Q5_K_S Model** (`merged-sci-model-q5_k_s.gguf`): Very good quality, slightly more compressed |
| - **Q4_K_M Model** (`merged-sci-model-q4_k_m.gguf`): Good quality, smallest size, recommended for resource-constrained environments |
| |
| All models use the **ChatML** template format and support up to **32K context length**. |
| |
| ## Intended Use |
| |
| This model is designed to: |
| - Provide SCI-specific information and guidance |
| - Answer questions about daily life with spinal cord injuries |
| - Offer practical advice for common SCI challenges |
| - Support the SCI community with contextually appropriate responses |
| |
| ## Limitations |
| |
| - This model is for informational purposes only and should not replace professional medical advice |
| - Always consult with healthcare providers for medical decisions |
| - The model may not have information about the latest medical developments |
| - Responses should be verified with medical professionals when making health-related decisions |
| |
| ## Direct Use |
| |
| This model can be used directly for: |
| - Educational purposes about spinal cord injuries |
| - Providing general information and support to the SCI community |
| - Research into specialized medical AI assistants |
| - Personal use by individuals seeking SCI-related information |
| |
| The model is designed to provide contextually appropriate responses that consider the unique challenges and medical realities of spinal cord injuries. |
| |
| ### Downstream Use |
| |
| This model can be fine-tuned further for: |
| - Integration into healthcare applications |
| - Specialized medical chatbots for rehabilitation centers |
| - Educational platforms for SCI awareness and training |
| - Research applications in medical AI |
| - Custom applications for SCI support organizations |
| |
| When used in downstream applications, implementers should: |
| - Maintain the medical disclaimer requirements |
| - Ensure proper supervision by medical professionals |
| - Implement appropriate safety measures and content filtering |
| - Validate outputs for medical accuracy in their specific use case |
| |
| ### Out-of-Scope Use |
| |
| This model should NOT be used for: |
| - **Medical diagnosis or treatment decisions** - Always consult healthcare professionals |
| - **Emergency medical situations** - Seek immediate professional medical help |
| - **Legal or financial advice** related to SCI cases |
| - **Replacement for professional medical consultation** |
| - **Clinical decision-making** without physician oversight |
| - **Applications targeting vulnerable populations** without proper safeguards |
| - **Commercial medical applications** without appropriate medical validation and oversight |
| |
| ## Bias, Risks, and Limitations |
| |
| ### Medical Limitations |
| - **Not a substitute for medical professionals**: All medical advice should be verified with qualified healthcare providers |
| - **Training data limitations**: May not include the most recent medical research or treatments |
| - **Individual variation**: SCI affects individuals differently; responses may not apply to all cases |
| - **Geographic bias**: Training data may be biased toward certain healthcare systems or regions |
| |
| ### Technical Limitations |
| - **Hallucination risk**: Like all language models, may generate plausible-sounding but incorrect information |
| - **Context limitations**: Limited by input context window and may not retain information across long conversations |
| - **Language limitations**: Primarily trained on English content |
| - **Update lag**: Cannot access real-time medical research or current events |
| |
| ### Bias Considerations |
| - **Training data bias**: Reflects biases present in source medical literature and online content |
| - **Demographic representation**: May not equally represent all demographics within the SCI community |
| - **Healthcare access bias**: May reflect biases toward certain types of healthcare systems |
| - **Severity bias**: May be more informed about certain types or severities of SCI |
| |
| ### Risk Mitigation |
| - Always include medical disclaimers when using this model |
| - Implement content filtering for harmful or dangerous advice |
| - Regular evaluation by medical professionals is recommended |
| - Monitor outputs for accuracy and appropriateness |
| |
| ## Recommendations |
| |
| Users should be aware of the following recommendations: |
| |
| **For Direct Users:** |
| - Always verify medical information with qualified healthcare professionals |
| - Use responses as educational/informational starting points, not definitive advice |
| - Be aware that individual SCI experiences vary significantly |
| - Seek immediate professional help for urgent medical concerns |
| |
| **For Developers/Implementers:** |
| - Implement clear medical disclaimers in any application using this model |
| - Provide easy access to professional medical resources alongside model responses |
| - Consider implementing content filtering for potentially harmful advice |
| - Regular review by medical professionals is strongly recommended |
| - Ensure compliance with relevant healthcare regulations (HIPAA, etc.) |
| |
| **For Healthcare Organizations:** |
| - Professional medical oversight is essential when implementing in clinical settings |
| - Regular validation of model outputs against current medical standards |
| - Integration should complement, not replace, professional medical consultation |
| - Staff training on AI limitations and appropriate use cases |
| |
| ## Training Details |
| |
| ### Training Data |
| |
| The training dataset consisted of 119,117 carefully curated entries focused on spinal cord injury information: |
| |
| **Domain Pretraining Data (35,779 entries):** |
| - Medical literature and research papers on SCI |
| - Educational materials from reputable SCI organizations |
| - Clinical guidelines and treatment protocols |
| - Rehabilitation and therapy documentation |
| - Patient education resources |
| |
| **Instruction Tuning Data (83,337 entries):** |
| - SCI-focused question-answer pairs |
| - Conversational examples with appropriate medical context |
| - Real-world scenarios and practical advice situations |
| - Educational Q&A formatted for instruction following |
| |
| All training data was filtered and curated to ensure: |
| - Sources from reputable medical organizations and healthcare professionals |
| - Content originally created or reviewed by medical professionals in the SCI field |
| - Appropriate tone and sensitivity for SCI community |
| - Removal of potentially harmful or dangerous advice |
| - Proper medical disclaimers and context |
| |
| **Note**: While the source materials were created by medical professionals, this model itself has not undergone independent medical validation. |
| |
| ### Training Procedure |
| |
| The model was trained using a two-phase approach with QLoRA (Quantized Low-Rank Adaptation): |
| |
| **Phase 1 - Domain Pretraining:** |
| - Focus: Medical terminology and SCI-specific knowledge |
| - Duration: 2 epochs (~8 hours) |
| - Data: 35,779 domain text entries |
| - Objective: Adapt base model to SCI medical domain |
| |
| **Phase 2 - Instruction Tuning:** |
| - Focus: Conversational abilities and response formatting |
| - Duration: 2 epochs (~12 hours) |
| - Data: 83,337 instruction-response pairs |
| - Objective: Teach appropriate response patterns and tone |
| |
| #### Preprocessing |
| |
| Training data underwent extensive preprocessing: |
| - Content sourced from materials created by healthcare professionals |
| - Sensitive content filtering and safety checks |
| - Standardized formatting for instruction-following |
| - Quality filtering to remove low-quality or inappropriate content |
| - Tokenization optimization for efficient training |
| |
| #### Training Hyperparameters |
| |
| - **Training regime:** 4-bit quantization with LoRA adapters (QLoRA) |
| - **Learning rate:** 2e-4 with cosine scheduling |
| - **LoRA rank:** 16 |
| - **LoRA alpha:** 32 |
| - **LoRA dropout:** 0.05 |
| - **Target modules:** q_proj, v_proj |
| - **Batch size:** 4 with gradient accumulation |
| - **Max sequence length:** 512 tokens |
| - **Optimizer:** AdamW with weight decay |
| |
| #### Speeds, Sizes, Times |
| |
| - **Total training time:** ~20 hours (8h Phase 1 + 12h Phase 2) |
| - **Hardware:** RTX 4070 Super (12GB VRAM) |
| - **Final model size:** 30MB (LoRA adapter only) |
| - **Base model size:** 7B parameters (not included in adapter) |
| - **Training throughput:** ~3.5 samples/second average |
| - **Memory usage:** 6-7GB VRAM during training |
| |
| ## Evaluation |
| |
| ### Testing Data, Factors & Metrics |
| |
| #### Testing Data |
| |
| The model was evaluated using: |
| - Held-out test set of SCI-related questions (500 samples) |
| - Manual review of response quality and appropriateness |
| - Comparative analysis against general-purpose models on SCI topics |
| - Assessment of domain-specific knowledge retention |
| |
| **Note**: Evaluation was conducted by the model developer, not independent medical professionals. |
| |
| #### Factors |
| |
| Evaluation considered multiple factors: |
| - **Medical accuracy**: Correctness of SCI-related information |
| - **Appropriateness**: Sensitivity and tone for SCI community |
| - **Contextual relevance**: Understanding of SCI-specific challenges |
| - **Safety**: Avoidance of harmful or dangerous advice |
| - **Completeness**: Comprehensive responses to complex questions |
| |
| #### Metrics |
| |
| - **Medical accuracy score**: Based on consistency with source medical literature (not independently validated) |
| - **Appropriateness rating**: Developer assessment of tone and sensitivity (4.2/5.0 subjective rating) |
| - **Response relevance**: SCI-specific context understanding (82% relevance score) |
| - **Safety compliance**: No obviously harmful medical advice detected in test samples |
| - **Response quality**: Perplexity improvements over base model for SCI domain |
| |
| ### Results |
| |
| **Quantitative Results:** |
| - 40% improvement in SCI domain perplexity over base model |
| - Responses demonstrate consistency with source medical literature |
| - 95% safety compliance (no obviously harmful medical advice detected) |
| - 82% average relevance score for SCI-specific contexts |
| |
| **Qualitative Results:** |
| - Responses demonstrate clear understanding of SCI terminology and concepts |
| - Appropriate tone and sensitivity for disability community |
| - Consistent inclusion of medical disclaimers |
| - Good balance between being helpful and cautious about medical advice |
| |
| **Limitations of Evaluation:** |
| - Evaluation conducted by model developer, not independent medical experts |
| - No formal clinical validation or testing with SCI patients |
| - Results based on consistency with training sources, not independent medical verification |
| |
| ## Environmental Impact |
| |
| Training carbon emissions estimated using energy consumption data: |
| |
| - **Hardware Type:** RTX 4070 Super (12GB VRAM) |
| - **Hours used:** ~20 hours total training time |
| - **Cloud Provider:** Local training (personal hardware) |
| - **Compute Region:** North America |
| - **Carbon Emitted:** Approximately 2.1 kg CO2eq (estimated based on local energy grid) |
| |
| The use of QLoRA significantly reduced training time and energy consumption compared to full fine-tuning methods, making this a relatively efficient training approach. |
| |
| ## Technical Specifications |
| |
| ### Model Architecture and Objective |
| |
| - **Base Architecture:** Mistral 7B transformer model |
| - **Adaptation Method:** QLoRA (Quantized Low-Rank Adaptation) |
| - **Objective:** Causal language modeling with SCI domain specialization |
| - **Quantization:** 4-bit precision for memory efficiency |
| - **LoRA Configuration:** Rank-16 adapters on attention projection layers |
| |
| ### Compute Infrastructure |
| |
| #### Hardware |
| |
| - **GPU:** NVIDIA RTX 4070 Super (12GB VRAM) |
| - **CPU:** Modern multi-core processor |
| - **RAM:** 32GB system memory |
| - **Storage:** NVMe SSD for fast data loading |
| |
| #### Software |
| |
| - **Framework:** Transformers 4.36+, PEFT 0.16.0 |
| - **Training:** QLoRA with bitsandbytes quantization |
| - **Environment:** Python 3.10+, PyTorch 2.0+, CUDA 12.1 |
| |
| ## Citation |
| |
| If you use this model in your research or applications, please cite: |
| |
| **BibTeX:** |
| ```bibtex |
| @misc{sci_assistant_2025, |
| title={SCI Assistant: A Specialized AI Assistant for Spinal Cord Injury Support}, |
| author={basiphobe}, |
| year={2025}, |
| howpublished={Hugging Face Model Repository}, |
| url={https://huggingface.co/basiphobe/sci-assistant} |
| } |
| ``` |
| |
| **APA:** |
| basiphobe. (2025). *SCI Assistant: A Specialized AI Assistant for Spinal Cord Injury Support*. Hugging Face. https://huggingface.co/basiphobe/sci-assistant |
| |
| ## Glossary |
| |
| **SCI**: Spinal Cord Injury - damage to the spinal cord that results in temporary or permanent changes in function |
| |
| **QLoRA**: Quantized Low-Rank Adaptation - an efficient fine-tuning method that reduces memory requirements |
| |
| **Domain Pretraining**: Training phase focused on learning domain-specific terminology and knowledge |
| |
| **Instruction Tuning**: Training phase focused on learning conversational patterns and response formatting |
| |
| **Perplexity**: A metric measuring how well a language model predicts text (lower is better) |
| |
| **LoRA**: Low-Rank Adaptation - parameter-efficient fine-tuning technique |
| |
| ## Model Card Authors |
| |
| **Primary Author:** basiphobe |
| **Model Development:** Individual research project for SCI community support |
| **Data Sources:** Curated from medical literature and educational materials created by healthcare professionals |
| **Validation Status:** Model has not undergone independent medical professional validation |
| |
| ## Model Card Contact |
| |
| For questions, issues, or feedback regarding this model: |
| - **Hugging Face:** https://huggingface.co/basiphobe/sci-assistant |
| - **Issues:** Please report issues through Hugging Face model repository |
| - **Medical Concerns:** Always consult qualified healthcare professionals |
| |
| **Important Note:** This model is provided for educational and informational purposes. Always seek professional medical advice for health-related questions and decisions. |
| |
| ### Framework versions |
| |
| - PEFT 0.16.0 |
| |
