This role is centered around the development of our custom-built "Ghost" robot, designed specifically to automate retail operations in Japanese convenience stores. The robot performs pick-and-place operations without human intervention, aiming to achieve a high level of autonomy and operational reliability.
The core responsibility of this role is to improve and expand the robot's perception capabilities—particularly through the development of vision-based systems that enable it to understand and act within complex, dynamic store environments. This includes leveraging sensor data (e.g., RGB, depth, and other modalities) collected from robots already deployed in real-world settings to enhance the performance of the perception system. The work involves researching, refining, and implementing new perception algorithms to further improve success rates, while also ensuring that deployed models are optimized and compressed to run efficiently on the system’s constrained hardware.
We strongly believe that our ghost robot has the potential to reshape the retail landscape in Japan and beyond, and we would be thrilled to work with someone who shares that vision and excitement.
Responsibilities
- Contribute to the development of an automation system using custom hardware by building sophisticated software for the perception system, incorporating deep learning, computer vision, and control algorithms.
- Collaborate with other automation engineers to meet priorities and goals set by the executive team.
- Break down abstract goals into actionable tasks and manage your own timeline to achieve company objectives.
- Analyze problems and data accurately, and deploy robust and safe systems within the given timelines.
- Collect data, analyze outcomes, and continuously improve algorithms with a focus on quantitative performance.
Requirements
- 3+ years of experience developing perception systems or equivalent experience, including academic research.
- Quick and accurate problem-solving skills.
- Strong debugging and issue-monitoring capabilities.
- Extensive experience in low-level computer vision, machine learning, and deep learning, including implementing state-of-the-art methods from scratch.
- Proficiency in Python and commonly used libraries, such as PyTorch, OpenCV, NumPy, SciKit. Experience in libraries such as Open3D, Point Cloud Library, MMDetection, FastAI etc. are not mandatory but considered an advantage.
- Experience working with RGB and depth sensors (or multi-sensor setups like RGB + LiDAR).
- Familiarity with data augmentation techniques.
- Proven track record in customizing deep learning applications, such as:
○ Object detection/segmentation
○Keypoint detection
○3D pose estimation and reconstruction
- Experience with model performance optimization techniques (speed, latency, …), for example:
○ Network layer optimization
○ TensorRT / TorchScript
Preferred Skills
- Proficiency in modern C++ (C++11, C++14, or newer)
- Experience with ROS
- Knowledge of camera calibration and hand-eye calibration
- Research experience in computer vision, AI, or robotics (e.g., publications at CVPR, ICRA)
- Experience in synthetic dataset generation and domain randomization techniques using Blender, OpenGL, Isaac Sim, MuJoCo, etc.
- Experience working with manipulators
- Background in startups or fast-paced environments
- Engineering experience in one or more of the following:
○Motion planning and execution
○Signal processing
○Network communication (e.g., pub/sub, req/rep)
