In this work, we investigate whether it is possible to distinguish conversational interac- tions from observing human motion alone, in particular subject specific gestures in 3D. We adopt Kinect sensors to obtain 3D displacement and velocity measurements, followed by wavelet decomposition to extract low level temporal features. These features are then generalized to form a visual vocabulary that can be further generalized to a set of topics from temporal distributions of visual vocabulary. A subject specific supervised learning approach based on Random Forests is used to classify the testing sequences to seven dif- ferent conversational scenarios. These conversational scenarios concerned in this work have rather subtle differences among them. Unlike typical action or event recognition, each interaction in our case contain many instances of primitive motions and actions, many of which are shared among different conversation scenarios. That is the interactions we are concerned with are not micro or instant events, such as hugging and high-five, but rather interactions over a period of time that consists rather similar individual motions, micro actions and interactions. We believe this is among one of the first work that is devoted to subject specific conversational interaction classification using 3D pose features and to show this task is indeed possible.
J. Deng, X. Xie, and B. Daubney, A bag of words approach to subject specific 3D human pose interaction classification with random decision forests, Graphical Models, Volume 76, Issue 3, Pages 162–171, May 2014.
More details can be found at the Swansea Vision website.
In this article, a new level set model is proposed for the segmentation of biomedical images. The image energy of the proposed model is derived from a robust image gradient feature which gives the active contour a global representation of the geometric configuration, making it more robust in dealing with image noise, weak edges, and initial configurations. Statistical shape information is incorporated using nonparametric shape density distribution, which allows the shape model to handle relatively large shape variations. The segmentation of various shapes from both synthetic and real images depict the robustness and efficiency of the proposed method.
S. Y. Yeo, X. Xie, I. Sazonov, and P. Nithiarasu, Segmentation of biomedical images using active contour model with robust image feature and shape prior, International Journal for Numerical Methods in Biomedical Engineering, Volume 30, Issue 2, pages 232–248, February 2014.
More details can be found at the Swansea Vision website.
We address the two inherently related problems of segmentation and interpolation of 3D and 4D sparse data and propose a new method to integrate these stages in a level set framework.
The interpolation process uses segmentation information rather than pixel intensities for increased robustness and accuracy. The method supports any spatial configurations of sets of 2D slices having arbitrary positions and orientations. We achieve this by introducing a new level set scheme based on the interpolation of the level set function by radial basis functions. The proposed method is validated quantitatively and/or subjec- tively on artificial data and MRI and CT scans and is compared against the traditional sequential approach, which interpolates the images first, using a state-of-the-art image interpolation method, and then segments the interpolated volume in 3D or 4D. In our experiments, the proposed framework yielded similar segmentation results to the sequential approach but provided a more robust and accurate interpolation. In particular, the interpolation was more satisfactory in cases of large gaps, due to the method taking into account the global shape of the object, and it recovered better topologies at the extremities of the shapes where the objects disappear from the image slices. As a result, the complete integrated framework provided more satisfactory shape reconstructions than the sequential approach.
A. Paiement, M. Mirmehdi, X. Xie, and M. Hamilton, Integrated Segmentation and Interpolation of Sparse Data, IEEE Transactions on Image Processing (T-IP), volume 23, issue 1, pages 110-125, January 2014.
More details can be found at the Swansea Vision website.
Video search interface
Traditional sketch-based image or video search systems rely on machine learning concepts as their core technology. However, in many applications, machine learning alone is impractical since videos may not be semantically annotated sufficiently, there may be a lack of suitable training data, and the search requirements of the user may frequently change for different tasks. In this work, we develop a visual analytics systems that overcomes the shortcomings of the traditional approach. We make use of a sketch-based interface to enable users to specify search requirement in a flexible manner without depending on semantic annotation. We employ active machine learning to train different analytical models for different types of search requirements. We use visualization to facilitate knowledge discovery at the different stages of visual analytics. This includes visualizing the parameter space of the trained model, visualizing the search space to support interactive browsing, visualizing candidature search results to support rapid interaction for active learning while minimizing watching videos, and visualizing aggregated information of the search results. We demonstrate the system for searching spatio-temporal attributes from sports video to identify key instances of the team and player performance.
Phil A. Legg, David H. S. Chung, Matt L. Parry, Rhodri Bown, Mark W. Jones, Iwan W. Griffiths, Min Chen.
IEEE Transactions on Visualization and Computer Graphics, 19(12), 2109-2118.
Path traced, 16 samples using Probabilistic illumination-aware filtering
Noise removal for Monte Carlo global illumination rendering is a well known problem, and has seen significant attention from image-based filtering methods. However, many state of the art methods breakdown in the presence of high frequency features, complex lighting and materials. In this work we present a probabilistic image based noise removal and irradiance filtering framework that preserves this high frequency detail such as hard shadows and glossy reflections, and imposes no restrictions on the characteristics of the light transport or materials. We maintain per-pixel clusters of the path traced samples and, using statistics from these clusters, derive an illumination aware filtering scheme based on the discrete Poisson probability distribution. Furthermore, we filter the incident radiance of the samples, allowing us to preserve and filter across high frequency and complex textures without limiting the effectiveness of the filter.
Ian C. Doidge and Mark W. Jones.
CGI 2013, The Visual Computer 29(6-8),707-616, 2013. The final publication is available at www.springerlink.com.
This paper presents a novel approach to detecting and preserving fine illumination structure within photon maps. Data derived from each photon’s primal trajectory is encoded and used to build a high-dimensional kd-tree. Incorporation of these new parameters allows for precise differentiation between intersecting ray envelopes, thus minimizing detail degradation when combined with photon relaxation. We demonstrate how parameter-aware querying is beneficial in both detecting and removing noise. We also propose a more robust structure descriptor based on principal components analysis that better identifies anisotropic detail at the sub-kernel level.We illustrate the effectiveness of our approach in several example scenes and show significant improvements when rendering complex caustics compared to previous methods.
Ben Spencer and Mark W. Jones
Computer Graphics Forum, Volume 32, Issue 2pt1, pages 83–92, May 2013. [doi]
Best paper, Eurographics 2013.
Stream compaction is an important parallel computing primitive that produces a reduced (compacted) output stream consisting of only valid elements from an input stream containing both invalid and valid elements. Computing on this compacted stream rather than the mixed input stream leads to improvements in performance, load balancing, and memory footprint. Stream compaction has numerous applications in a wide range of domains: e.g., deferred shading, isosurface extraction, and surface voxelization in computer graphics and visualization. We present a novel In-Kernel stream compaction method, where compaction is completed before leaving an operating kernel. This contrasts with conventional parallel compaction methods that require leaving the kernel and running a prefix sum kernel followed by a scatter kernel. We apply our compaction methods to ray-tracing-based visualization of volumetric data. We demonstrate that the proposed In-Kernel Compaction outperforms the standard out-of-kernel Thrust parallel-scan method for performing stream compaction in this real-world application. For the data visualization, we also propose a novel multi-kernel ray-tracing pipeline for increased thread coherency and show that it outperforms a conventional single-kernel approach.
D. M. Hughes, I. S. Lim, M. W. Jones, A. Knoll and B. Spencer
Computer Graphics Forum, 2013, 32(6), 178-188. [doi]
In written and spoken communications, figures of speech (e.g., metaphors and synecdoche) are often used as an aid to help convey abstract or less tangible concepts. However, the benefits of using rhetorical illustrations or embellishments in visualization have so far been inconclusive. In this work, we report an empirical study to evaluate hypotheses that visual embellishments may aid memorization, visual search and concept comprehension. One major departure from related experiments in the literature is that we make use of a dualtask methodology in our experiment. This design offers an abstraction of typical situations where viewers do not have their full attention focused on visualization (e.g., in meetings and lectures). The secondary task introduces “divided attention”, and makes the effects of visual embellishments more observable. In addition, it also serves as additional masking in memory-based trials. The results of this study show that visual embellishments can help participants better remember the information depicted in visualization. On the other hand, visual embellishments can have a negative impact on the speed of visual search. The results show a complex pattern as to the benefits of visual embellishments in helping participants grasp key concepts from visualization.
Rita Borgo, Alfie Abdul-Rahman, Farhan Mohamed, Philip W. Grant, Irene Reppa, Luciano Floridi, and Min Chen
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 18, NO. 12, DECEMBER 2012
Today real-time sports performance analysis is a crucial aspect of matches in many major sports. For example, in soccer and rugby, team analysts may annotate videos during the matches by tagging specific actions and events, which typically result in some summary statistics and a large spreadsheet of recorded actions and events. To a coach, the summary statistics (e.g., the percentage of ball possession) lacks sufficient details, while reading the spreadsheet is time-consuming and making decisions based on the spreadsheet in real-time is thereby impossible. In this paper, we present a visualization solution to the current problem in real-time sports performance analysis. We adopt a glyph-based visual design to enable coaching staff and analysts to visualize actions and events “at a glance”. We discuss the relative merits of metaphoric glyphs in comparison with other types of glyph designs in this particular application. We describe an algorithm for managing the glyph layout at different spatial scales in interactive visualization. We demonstrate the use of this technical approach through its application in rugby, for which we delivered the visualization software, MatchPad, on a tablet computer. The MatchPad was used by the Welsh Rugby Union during the Rugby World Cup 2011. It successfully helped coaching staff and team analysts to examine actions and events in detail whilst maintaining a clear overview of the match, and assisted in their decision making during the matches. It also allows coaches to convey crucial information back to the players in a visually-engaging manner to help improve their performance.
Phil A. Legg, David H. S. Chung, Matthew L. Parry, Mark W. Jones, Rhys Long, Iwan W. Griffiths and Min Chen.
Eurovis 2012, Computer Graphics Forum 31(3), 1255-1264, 2012. [doi] [BibTeX]
In this paper we seek to eliminate the noise caused by caustic paths during progressive Monte Carlo path tracing. We employ a filtering strategy over path space, handling each subspace using specialised derivations of path tracing and progressive photon mapping. Evaluating diffuse paths with path tracing allows the use of sample strati cation over both pixels and the image as a whole, whilst sharp detailed caustics are produced using progressive photon mapping. This is an efficient, low noise progressive algorithm with vanishing bias combining the advantages of both Monte Carlo methods, and particle tracing.
Ian C. Doidge, Mark W. Jones and Ben Mora.
CGI 2012, The Visual Computer 28(6-8), 603-612, 2012: The final publication is available at www.springerlink.com. [doi] [BibTeX]