The NVIDIA® Isaac Software Development Kit (SDK) is a developer toolbox for accelerating the development and deployment of AI-powered robots. The SDK includes the Isaac Robot Engine, packages with high-performance robotics algorithms, hardware reference applications and powerful simulation platform, Isaac Sim. The SDK accelerates robot development for manufacturers, researchers and startups by making it easier to add Artificial Intelligence (AI) for perception and navigation into modern day robots.
Isaac SDK capabilities:
- Deep neural networks and GPU accelerated algorithms for robotics.
- Develop in simulation and deploy easily in reality.
- Integration of NVIDIA AI capabilities for robotic applications.
- Easy bring up of sensors used for robotics applications.
- Visualization of robot development in real and virtual worlds.
- Interface with robotic platforms such as cobots (collaborative robots), AMRs (autonomous mobile robots), and other service/industrial robots.
The Isaac Robot Engine is a software framework to easily build modular robotics applications. It enables high-performance data processing and deep learning for intelligent robots. Robotics applications developed on Isaac Robot Engine can seamlessly run on edge devices like the NVIDIA® Jetson AGX Xavier™ and NVIDIA® Jetson Nano™, as well as on a workstation with a discrete NVIDIA® GPU.
- The Robot Engine allows developers to break down a complex robotics use case into smaller building blocks.
- Customize Isaac SDK features to fit your use case by configuring pre-packaged components.
- Easily add new features by developing custom components.
- Other capabilities include experimental message recording and replay of sensor data or message passing over network sockets.
- Applications consist of multiple nodes and use inter-node communication to exchange data between them.
- Avoid host-device memory copies and increase the performance of your application by attaching CUDA buffer objects to messages.
- Group nodes into subgraphs and effectively combine them into a robotics application.
- The Isaac Robot Engine also comes with a customizable visualization framework to create visualization for variable plots, drawings, camera image overlays, or 3D scenes.
- Developers can use Isaac WebSight to inspect and debug their robotics application in a web browser.
Advanced Build System
- Isaac SDK uses Bazel for tracking dependencies, building source code, testing and packaging of robotics applications.
- Automatically pull external dependencies such as OpenCV, ROS or other robotics open source libraries.
- Partners can create their own workspaces to add sensor support or new algorithms to Isaac SDK. An example is available on GitHub
What's New for Robot Engine in 2019.2
- Introducing distributed workspaces for Isaac SDK users and partners to create compatible, easily manageable workspaces and contribute to the growing Isaac community. Velodyne Lidar example available. Get the example from Isaac GitHub repository.
GEMS are modular capabilities for sensing, planning, or actuation that can be easily plugged into a robotics application. For example, developers can add obstacle detection, stereo depth estimation, or human speech recognition to enrich their robot use cases. Similarly, developers can use the Isaac navigation stack, a collection of modules to build a robotics navigation stack, to their robot. GEMS capabilities currently supported by Isaac SDK:
- Stereo Depth, Stereo Visual Odometry, Object/AprilTag Detection, Path Seg DNN, CUDA Orb, Superpixels
- Lidar based Global Localization, LQR Path Planning, Support for non-circular robot bases
IX - Human Machine Interaction
- Stereo Camera, Structured Light Depth Camera, Velodyne VLP16 lidar, IMU
What's New for GEMS in 2019.2
- New capabilities for perception tasks:
- Path Segmentation DNN
- CUDA ORB Features
- CUDA Superpixels
- New navigation capability with support for non-circular robot bases
- New sensor support for Livox Lidar
Isaac Sim is a virtual robotics laboratory and a high-fidelity 3D world simulator. It accelerates research, design, and development in robotics by reducing cost and risk. Developers can quickly and easily train and test their robots in detailed, highly realistic scenarios.
Developers can use virtual robots with simulated sensors (RGB, stereo, depth, segmentation, LIDAR, IMU) in Isaac Sim to test their applications in a high-fidelity simulation environment. Once tested, applications can be deployed to NVIDIA® Jetson AGX Xavier™, Jetson™TX2, or Jetson Nano™ running on physical robots.
Robots in Isaac Sim are tightly coupled to the tools and frameworks in Isaac SDK, enabling easy transfer of algorithms and data between physical and virtual robots.
Using synthetic data and controlling the randomization parameters in a scene, developers can create huge customized data sets. In Isaac Sim, developers can seamlessly randomize lighting, object materials, object colors, object poses and camera properties.
The most common robot model is URDF. Isaac Sim has an embedded URDF loader which allows you to load the URDF model of your robot into Isaac Sim and simulate its joints and movements. The URDF models for several mobile bases and manipulators have been tested in Isaac Sim.
What's New for Simulation in 2019.2
- Introducing Isaac Sim support for Unity 3D Game Engine, popular for building scenarios without the need for in-house 3D expertise.
REFERENCE DESIGN & APPLICATIONS
Kaya the robot to get started with robotics
People who would like to get started with Isaac SDK can build their own small robot platform using Kaya reference robot platform.
Kaya can be built using off-the-shelf components and 3D printed parts. The low-cost Kaya robot takes advantage of the new NVIDIA® Jetson Nano™. Samples applications built for the platform include April tag detection, obstacle avoidance, remote operation, object classification based on Yolo and much more. Some of the included apps are:
Carter the delivery robot
Carter is an Isaac SDK reference robot platform for autonomous indoor delivery and logistics based on NVIDIA Jetson platform. Developers can build their own Carter robot using sample applications. With NVIDIA Isaac SDK, developers can add capabilities for localization, autonomous navigation, map edition, path planning and state machines. Some of the included apps are:
- Map Creation
- Map Waypoint