Skip to content
@neuralmagic

Neural Magic

Neural Magic helps developers in accelerating machine learning performance using automated model sparsification techniques and inference technologies.

Pinned Loading

  1. nm-vllm-certs nm-vllm-certs Public

    General Information, model certifications, and benchmarks for nm-vllm enterprise distributions

    7 1

  2. deepsparse deepsparse Public

    Sparsity-aware deep learning inference runtime for CPUs

    Python 3k 173

  3. sparseml sparseml Public

    Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

    Python 2.1k 148

  4. docs docs Public

    Top-level directory for documentation and general content

    MDX 120 7

  5. sparsezoo sparsezoo Public

    Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

    Python 372 25

  6. guidellm guidellm Public

    Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

    Python 174 12

Repositories

Showing 10 of 59 repositories
  • vllm Public Forked from vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    neuralmagic/vllm’s past year of commit activity
    Python 6 Apache-2.0 4,881 0 19 Updated Dec 9, 2024
  • upstream-transformers Public Forked from huggingface/transformers

    🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

    neuralmagic/upstream-transformers’s past year of commit activity
    Python 1 Apache-2.0 27,638 0 0 Updated Dec 9, 2024
  • nm-actions Public

    Neural Magic GHA

    neuralmagic/nm-actions’s past year of commit activity
    Python 0 Apache-2.0 0 0 3 Updated Dec 9, 2024
  • nm-vllm-certs Public

    General Information, model certifications, and benchmarks for nm-vllm enterprise distributions

    neuralmagic/nm-vllm-certs’s past year of commit activity
    7 1 1 0 Updated Dec 9, 2024
  • guidellm Public

    Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

    neuralmagic/guidellm’s past year of commit activity
    Python 174 Apache-2.0 12 10 9 Updated Dec 6, 2024
  • compressed-tensors Public

    A safetensors extension to efficiently store sparse quantized tensors on disk

    neuralmagic/compressed-tensors’s past year of commit activity
    Python 52 Apache-2.0 2 2 14 Updated Dec 6, 2024
  • quant_kernel_benchmarks Public

    Benchmarking code for running quantized kernels from vLLM and other libraries

    neuralmagic/quant_kernel_benchmarks’s past year of commit activity
    Python 2 0 0 0 Updated Dec 3, 2024
  • yolov5 Public Forked from ultralytics/yolov5

    YOLOv5 in PyTorch > ONNX > CoreML > TFLite

    neuralmagic/yolov5’s past year of commit activity
    Python 20 GPL-3.0 16,650 0 2 Updated Dec 2, 2024
  • lm-evaluation-harness Public Forked from EleutherAI/lm-evaluation-harness

    A framework for few-shot evaluation of language models.

    neuralmagic/lm-evaluation-harness’s past year of commit activity
    Python 2 MIT 1,946 0 1 Updated Nov 27, 2024
  • docs Public

    Top-level directory for documentation and general content

    neuralmagic/docs’s past year of commit activity
    MDX 120 7 0 4 Updated Nov 25, 2024