- 标题
- 摘要
- 关键词
- 实验方案
- 产品
过滤筛选
- 2018
- hollows classification
- RGB-D camera
- domain similarity
- depth image inpainting
- color consistency
- Optoelectronic Information Science and Engineering
- Chongqing University of Posts and Telecommunications
-
SCN: Switchable Context Network for Semantic Segmentation of RGB-D Images
摘要： Context representations have been widely used to profit semantic image segmentation. The emergence of depth data provides additional information to construct more discriminating context representations. Depth data preserves the geometric relationship of objects in a scene, which is generally hard to be inferred from RGB images. While deep convolutional neural networks (CNNs) have been successful in solving semantic segmentation, we encounter the problem of optimizing CNN training for the informative context using depth data to enhance the segmentation accuracy. In this paper, we present a novel switchable context network (SCN) to facilitate semantic segmentation of RGB-D images. Depth data is used to identify objects existing in multiple image regions. The network analyzes the information in the image regions to identify different characteristics, which are then used selectively through switching network branches. With the content extracted from the inherent image structure, we are able to generate effective context representations that are aware of both image structures and object relationships, leading to a more coherent learning of semantic segmentation network. We demonstrate that our SCN outperforms state-of-the-art methods on two public datasets.
关键词： Context representation,convolutional neural network (CNN),RGB-D images,semantic segmentation
更新于2025-09-23 15:23:52
-
[IEEE 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA) - Barranquilla, Colombia (2018.11.1-2018.11.3)] 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA) - A proposal for a SoC FPGA-based image processing in RGB-D sensors for robotics applications
摘要： The current robots follow clear, repetitive and logical instructions, but generally, they have problems in managing unstructured environments and reacting dynamically to these. Thus, modern robots require improved vision systems capable of obtaining information about such environments at a high acquisition rate and with high processing speeds. The growing demand for robotic platforms, both industrial and mobile, has greatly boosted the development of advanced vision systems. A weak point of traditional computer vision is that it depends on algorithms executed on a computer or server connected to the robot, often involving the need for high computing resources. Therefore, much of the efforts of the last decades have been focused on the improvement of those algorithms. Nevertheless, when the limit of traditional software processing systems (PCs, microcontrollers and microprocessors) is reached, it is necessary to migrate to a more versatile platform -which generally leads to hardware solutions-. The HW/SW design is possible because of high-frequency bridges between the Hard-Processor System (HPS) and the FPGA. Commonly, the most demanding tasks of the image processing are made in the FPGA, whereas the HPS handles the processed data and performs the high-level control function. This work presents a proposal for HW/SW integration using a SoC FPGA, for the images processing provided by the Intel Realsense?3D camera (an RGB-D sensor). This approach seeks to enhance the streamlining and ?ltering stages to obtain faster results compared to a traditional system.
关键词： SoC FPGA,Intel RealSense R200,image processing,RGB-D
更新于2025-09-23 15:23:52
-
[IEEE 2018 24th International Conference on Pattern Recognition (ICPR) - Beijing, China (2018.8.20-2018.8.24)] 2018 24th International Conference on Pattern Recognition (ICPR) - Accurate 3-D Reconstruction with RGB-D Cameras using Depth Map Fusion and Pose Refinement
摘要： Depth map fusion is an essential part in both stereo and RGB-D based 3-D reconstruction pipelines. Whether produced with a passive stereo reconstruction or using an active depth sensor, such as Microsoft Kinect, the depth maps have noise and may have poor initial registration. In this paper, we introduce a method which is capable of handling outliers, and especially, even significant registration errors. The proposed method first fuses a sequence of depth maps into a single non-redundant point cloud so that the redundant points are merged together by giving more weight to more certain measurements. Then, the original depth maps are re-registered to the fused point cloud to refine the original camera extrinsic parameters. The fusion is then performed again with the refined extrinsic parameters. This procedure is repeated until the result is satisfying or no significant changes happen between iterations. The method is robust to outliers and erroneous depth measurements as well as even significant depth map registration errors due to inaccurate initial camera poses.
关键词： point cloud,3-D reconstruction,RGB-D cameras,pose refinement,depth map fusion,registration errors
更新于2025-09-23 15:23:52
-
[ACM Press the 2nd International Conference - Tianjin, China (2018.09.19-2018.09.21)] Proceedings of the 2nd International Conference on Biomedical Engineering and Bioinformatics - ICBEB 2018 - 3D Human Pose Estimation from RGB+D Images with Convolutional Neural Networks
摘要： In this paper, we explore 3D human pose estimation on the RGB+D images. While many researchers try to directly predict 3D pose from single RGB image, we propose a simple framework that could predict 3D pose predictions with the RGB image and depth image. Our approach is based on two aspects. On the one hand, we predicted accurate 2D joint locations from RGB image by applying the stacked hourglass networks based on the improved residual architecture. On the other hand, in view of obtained 2D joint locations, we could estimate 3D pose with the depth after calculating depth image patches. In general, compared with the state-of-the-art approaches, our model achieves signification improvement on benchmark dataset.
关键词： Deep Learning,Human Pose Estimation,RGB+D Images
更新于2025-09-23 15:23:52
-
Comparing RGB-D Sensors for Close Range Outdoor Agricultural Phenotyping
摘要： Phenotyping is the task of measuring plant attributes for analyzing the current state of the plant. In agriculture, phenotyping can be used to make decisions concerning the management of crops, such as the watering policy, or whether to spray for a certain pest. Currently, large scale phenotyping in fields is typically done using manual labor, which is a costly, low throughput process. Researchers often advocate the use of automated systems for phenotyping, relying on the use of sensors for making measurements. The recent rise of low cost, yet reasonably accurate, RGB-D sensors has opened the way for using these sensors in field phenotyping applications. In this paper, we investigate the applicability of four different RGB-D sensors for this task. We conduct an outdoor experiment, measuring plant attribute in various distances and light conditions. Our results show that modern RGB-D sensors, in particular, the Intel D435 sensor, provides a viable tool for close range phenotyping tasks in fields.
关键词： INTEL D-435,RGB-D sensors,sensors in agriculture,INTEL SR300,empirical analysis,Microsoft Kinect,phenotyping,ORBBEC ASTRA S
更新于2025-09-23 15:22:29
-
[IEEE 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) - Singapore, Singapore (2018.11.18-2018.11.21)] 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) - Bi-Manual Articulated Robot Teleoperation using an External RGB-D Range Sensor
摘要： In this paper, we present an implementation of a bi-manual teleoperation system, controlled by a human through three-dimensional (3D) skeleton extraction. The input data is given from a cheap RGB-D range sensor, such as the ASUS Xtion PRO. To achieve this, we have implemented a 3D version of the impressive OpenPose package, which was recently developed. The first stage of our method contains the execution of the OpenPose Convolutional Neural Network (CNN), using a sequence of RGB images as input. The extracted human skeleton pose localisation in two-dimensions (2D) is followed by the mapping of the extracted joint location estimations into their 3D pose in the camera frame. The output of this process is then used as input to drive the end-pose of the robotic hands relative to the human hand movements, through a whole-body inverse kinematics process in the Cartesian space. Finally, we implement the method as a ROS wrapper package and we test it on the centaur-like CENTAURO robot. Our demonstrated task is of a box and lever manipulation in real-time, as a result of a human task demonstration.
关键词： bi-manual robot,teleoperation,skeleton extraction,CENTAURO robot,OpenPose,inverse kinematics,RGB-D sensor
更新于2025-09-23 15:22:29
-
Dynamic Non-Rigid Objects Reconstruction with a Single RGB-D Sensor
摘要： This paper deals with the 3D reconstruction problem for dynamic non-rigid objects with a single RGB-D sensor. It is a challenging task as we consider the almost inevitable accumulation error issue in some previous sequential fusion methods and also the possible failure of surface tracking in a long sequence. Therefore, we propose a global non-rigid registration framework and tackle the drifting problem via an explicit loop closure. Our novel scheme starts with a fusion step to get multiple partial scans from the input sequence, followed by a pairwise non-rigid registration and loop detection step to obtain correspondences between neighboring partial pieces and those pieces that form a loop. Then, we perform a global registration procedure to align all those pieces together into a consistent canonical space as guided by those matches that we have established. Finally, our proposed model-update step helps fixing potential misalignments that still exist after the global registration. Both geometric and appearance constraints are enforced during our alignment; therefore, we are able to get the recovered model with accurate geometry as well as high fidelity color maps for the mesh. Experiments on both synthetic and various real datasets have demonstrated the capability of our approach to reconstruct complete and watertight deformable objects.
关键词： 3D reconstruction,non-rigid reconstruction,RGB-D sensor
更新于2025-09-23 15:22:29
-
[IEEE 2018 International Conference on 3D Vision (3DV) - Verona (2018.9.5-2018.9.8)] 2018 International Conference on 3D Vision (3DV) - Fusion++: Volumetric Object-Level SLAM
摘要： We propose an online object-level SLAM system which builds a persistent and accurate 3D graph map of arbitrary reconstructed objects. As an RGB-D camera browses a cluttered indoor scene, Mask-RCNN instance segmentations are used to initialise compact per-object Truncated Signed Distance Function (TSDF) reconstructions with object size-dependent resolutions and a novel 3D foreground mask. Reconstructed objects are stored in an optimisable 6DoF pose graph which is our only persistent map representation. Objects are incrementally re?ned via depth fusion, and are used for tracking, relocalisation and loop closure detection. Loop closures cause adjustments in the relative pose estimates of object instances, but no intra-object warping. Each object also carries semantic information which is re?ned over time and an existence probability to account for spurious instance predictions.
关键词： SLAM,object-level mapping,Mask-RCNN,3D reconstruction,RGB-D
更新于2025-09-23 15:21:01
-
Canonical Correlation Analysis Regularization: An Effective Deep Multi-View Learning Baseline for RGB-D Object Recognition
摘要： Object recognition methods based on multi-modal data, color plus depth (RGB-D), usually treat each modality separately in feature extraction, which neglects implicit relations between two views and preserves noise from any view to the ?nal representation. To address these limitations, we propose a novel Canonical Correlation Analysis (CCA)-based multi-view Convolutional Neural Network (CNNs) framework for RGB-D object representation. The RGB and depth streams process corresponding images respectively, then are connected by CCA module leading to a common-correlated feature space. In addition, to embed CCA into deep CNNs in a supervised manner, two different schemes are explored. One considers CCA as a regularization term adding to the loss function (CCAR). However, solving CCA optimization directly is neither computationally ef?cient nor compatible with the mini-batch based stochastic optimization. Thus, we further propose an approximation method of CCA regularization (ACCAR), using the obtained CCA projection matrices to replace the weights of feature concatenation layer at regular intervals. Such a scheme enjoys bene?ts of full CCA regularization and is ef?cient by amortizing its cost over many training iterations. Experiments on benchmark RGB-D object recognition datasets have shown that the proposed methods outperform most existing methods using the very same of their network architectures.
关键词： Deep learning,Canonical Correlation Analysis,Multi-view feature learning,RGB-D object recognition
更新于2025-09-23 15:21:01
-
[IEEE 2019 International Topical Meeting on Microwave Photonics (MWP) - Ottawa, ON, Canada (2019.10.7-2019.10.10)] 2019 International Topical Meeting on Microwave Photonics (MWP) - Multi Pole Microwave Filtering using Brillouin Scattering in Silicon
摘要： Acquiring general material appearance with hand-held consumer RGB-D cameras is dif?cult for casual users, due to the inaccuracy in reconstructed camera poses and geometry, as well as the unknown lighting that is coupled with materials in measured color images. To tackle these challenges, we present a novel technique for estimating the spatially varying isotropic surface re?ectance, solely from color and depth images captured with an RGB-D camera under unknown environment illumination. The core of our approach is a joint optimization, which alternates among solving for plausible camera poses, materials, the environment lighting and normals. To re?ne camera poses, we exploit the rich spatial and view-dependent variations of materials, treating the object as a localization-self-calibrating model. To recover the unknown lighting, measured color images along with the current estimate of materials are used in a global optimization, ef?ciently solved by exploiting the sparsity in the wavelet domain. We demonstrate the substantially improved quality of estimated appearance on a variety of daily objects.
关键词： RGB-D camera,joint optimization,spatially varying BRDF
更新于2025-09-23 15:19:57