Locally refinable gradient meshes supporting branching and sharp colour transitions

Locally refinable gradient meshes supporting branching and sharp colour transitions We present a local refinement approach for gradient meshes, a primitive commonly used in the design of vector illustrations with complex colour propagation. Local refinement allows the artist to add more detail only in the regions where it is needed, as opposed to global refinement which often clutters the workspace with undesired detail and potentially slows down the workflow. Moreover, in contrast to existing implementations of gradient mesh refinement, our approach ensures mathematically exact refinement. Additionally, we introduce a branching feature that allows for a wider range of mesh topologies, as well as a feature that enables sharp colour transitions similar to diffusion curves, which turn the gradient mesh into a more versatile and expressive vector graphics primitive. Keywords Vector graphics · Gradient meshes · Colour interpolation · Local refinement 1 Introduction the way gradient meshes behave in these software packages (see also Fig. 6), the overall concept is the same. In this Vector graphics provide an interesting alternative to raster paper we use the term traditional gradient mesh to refer to graphics, as they are resolution-independent and in gen- the primitive as implemented in these suites. eral easier to edit because they are composed of objects Upon using the traditional gradient mesh, it becomes clear (often called primitives) as opposed to coloured pixels. The that it comes with several limitations. The main shortcoming number of different primitives is rather limited, with the gra- is its fixed rectangular topology, making it difficult to design dient mesh as one of the most complex ones. This primitive complex objects and impossible to perform local refinement. facilitates the creation of intricate colour propagation over As is well-known in Computer Graphics, the latter aspect regular quadrilateral meshes, contributing to the possibility is a desirable feature [12]. Global refinement (propagating of designing photorealistic illustrations in vector format. through the entire row/column) often adds control points to Gradient meshes were introduced by Adobe Illustrator [1] the mesh where they are not needed, thereby cluttering the as the Mesh object. CorelDRAW soon followed [7] with a workspace and potentially slowing down the workflow. The similar tool called Mesh Fill. More recently, the primitive was conceptual difference between global and local refinement is added to the proposal for SVG 2 [27] and is now available in depicted in Fig. 1. In addition, the traditional gradient mesh Inkscape [14] as well. Although there are subtle differences in allows little to no control over the propagation of colour along curves and (as a consequence) in the interior, thus limiting the range of artistic styles that can be expressed with it. Electronic supplementary material The online version of this article It is our aim to create a more versatile and expressive prim- (https://doi.org/10.1007/s00371-018-1547-1) contains supplementary material, which is available to authorized users. itive, in particular from the perspective of manual creation and editing. Our contributions in this direction are: B Pieter J. Barendrecht p.j.barendrecht@rug.nl – Mathematically exact local refinement, which adds con- Bernoulli Institute, University of Groningen, Nijenborgh 9, 9747 Groningen, The Netherlands trol points only where they are required 123 950 P. J. Barendrecht et al. Fig. 3 Local refinement using Catmull–Clark subdivision. Vertices in the control net (orange) are generally not interpolated. Local refinement affects the valency of faces, which results in different limit surfaces (patch boundaries in black) Fig. 1 Global refinement (left), where each refinement propagates through an entire row/column, is the only form of refinement avail- 2 Related work able in existing implementations of the gradient mesh primitive. We propose a local refinement approach (right) The traditional gradient mesh primitive is conceptually straightforward. As such, the existing literature focuses on extensions or applications of the tool. The prior work can be classified into three different categories, discussing either the topology of the gradient mesh, aspects related to colour, or vectorisation, the (semi-) automatic creation of vector graphics from raster graphics. We conclude with a concise overview of alternative techniques. Considering the matter of restricted topology, in related fields subdivision surfaces are often used to model meshes of arbitrary topology. The catch in using them for gradient meshes is that neither interpolating subdivision schemes nor Fig. 2 Using a single gradient mesh to model a leek (control net top, approximating ones can be directly applied in a generally rendering bottom). Local refinement is used to express local details, the branching feature is used to create the non-rectangular topology satisfactory way. Using the former often results in colours desired for modelling this shape, and sharp colour transitions are used outside the gamut (which leads to flat or uniformly coloured to express depth spots), whereas the latter typically results in washed-out colours. The approaches taken by [17,31] use Loop’s approx- imating subdivision scheme for editing and vectorisation. – Branching, which enables the creation of non-rectangular Consequently, in practice the triangular elements in the mesh meshes by allowing branches, loops and (as a conse- need to be quite small for the intended colours to be approx- quence) holes imated closely enough (they are not interpolated), which – Assignment of colours to individual sectors of control in turn might complicate the editing process. This issue points, which facilitates sharp colour transitions is mostly avoided in [18] by combining augmented lin- ear ternary subdivision and Catmull–Clark’s approximating scheme supporting quad-dominant meshes of arbitrary topol- An example gradient mesh using all three novel features ogy. However, though colour is interpolated, by default the is shown in Fig. 2. geometry (represented by the control net) is not. Moreover, The structure of the remainder of this paper is as fol- like [17,31], the proposed method does not allow for local lows. In Sect. 2, we discuss and compare related work on refinement without changing the geometry or the colour prop- gradient meshes and alternative tools. We describe our inter- agation. In this subdivision setting, local refinement changes pretation of the gradient mesh primitive in Sect. 3, focusing the valency of faces and/or introduces vertices of irregular on geometry and colour blending. Refinement is discussed valencies, thereby changing the limit surface. See Fig. 3,cf. in detail in Sect. 4. User interaction (i.e., editing a gradient Fig. 1 (right). mesh) is considered in Sect. 5. Relevant details regarding the Finally, neither the PDF 2.0 [21] nor the SVG 2 standard implementation are described in Sect. 6. Branching and sharp supports the surface patches required to render subdivision colour transitions are described in Sect. 7 and are illustrated surfaces, which reduces the applicability of subdivision- using a number of examples (see also the supplementary based methods. video). Finally, we evaluate the advantages and limitations Methods not based on subdivision include the use of of our contributions in Sect. 8, and look ahead to future work curved Bézier triangles [28] for vectorisation. The approach in Sect. 9. touches only briefly on colour continuity across two trian- 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 951 gular elements, which is not sufficient for overall continuity. Related to diffusion curves, [19] associates shading pro- Although editing of the generated elements is discussed to files to the curves and subsequently converts the scene to a some extent, the method does not appear to support manual Catmull–Clark subdivision surface which can be rendered addition of elements. Finally, [16] considers vectorisation as directly. As it is subdivision-based, it comes with the same an application of their generalised barycentric coordinates drawbacks as mentioned above. for Hermite interpolation, which allows the use of highly irregular patch shapes. While interesting for vectorisation, the method does not seem suitable for intuitive manual cre- 3 The gradient mesh primitive ation of gradient meshes. To the best of our knowledge, no extensive comparison We now construct our interpretation of the gradient mesh regarding the use of different colour spaces for gradient primitive. First, we consider the geometrical aspects, after meshes has been published. However, preference for cer- which we focus on blending the colours defined on top of the tain colour spaces, including CIELAB, has been mentioned 2D geometry. In addition, “Appendix A” briefly discusses the before [18]. Instead, work on colour-related aspects focuses use of different colour spaces. on colour transfer, which discusses how colour characteris- As pointed out in [24], (semi-)transparency in vector tics from an existing (raster) image can be used to (re-)colour graphics is an important feature. In addition to the R, G and a gradient mesh [29,30]. B channels, a fourth channel A could be added to smoothly Most work primarily focuses on vectorisation. In [22], blend different alpha values assigned to control points. cubic Bézier patches are used to approximate individual To fully appreciate this feature, a layer system should be objects extracted from raster images. Interestingly, it also integrated to handle overlapping meshes. Although straight- discusses the need for (in this case, automatic) local refine- forward to implement, our framework currently does not ment to efficiently express local detail. However, smoothness support it as it adds little value from a scientific point of of the result is not guaranteed, and locally refined areas can- view. In the following, we adopt the RGB colour space. not be edited directly. A different approach is taken in [25], where the positions and other properties at control points of a 3.1 Geometry given (manually created) gradient mesh of rectangular topol- ogy can be optimised. In contrast, [15] proposes a completely In essence, the geometrical aspect consists of a 2D curve net- automatic approach, which is more generally applicable as it work in the xy-plane with (traditionally) the topology of a can vectorise objects with holes using a single mesh. How- rectangular grid. An example is shown in Fig. 1 (left). The ever, like [25], it uses rectangular meshes, and holes can only curves composing this network are commonly cubic curves. be formed by separating surface patches along a chain of Throughout the paper we use both Bézier and Hermite formu- curve segments, which restricts their shape. lations of these curves (and later surface patches). As such, Obtaining the desired topology using a (traditional) gra- each cubic curve C (t ) can be described as dient mesh can be quite a challenge. A common workflow is to use multiple gradient meshes to design the envisioned 3 object. Still, manual creation of complex shapes remains C (t ) = P B (t ) = P B(t ) (1) k k time-consuming. Diffusion curves [20] alleviate this task by k=0 allowing the user to draw outlines and open or closed inner = Q H (t ) = Q H (t ), (2) regions where changes in colour might occur, using Bézier k k k=0 or B-spline curves. Assigning (possibly different) colours to the two sides of the curves, the interior colour is determined where t ∈[0, 1]. Note that P are the Bézier control points using the principle of diffusion. From a mathematical point of and B (t ) the cubic Bernstein functions, Q the Hermite con- k k view, this is interpreted as solving a harmonic or biharmonic trols (points or tangent handles) and H (t ) the cubic Hermite [11] equation. In addition to the multigrid method used orig- functions. inally, existing approaches for solving these PDEs include As two different representations are considered, a way to FEM [6], BEM [26] and approximation by ray-tracing [5]. convert between them is desirable. Hermite control data Q Although the workflow is closer to the classical workflow can readily be obtained from the Bézier control data P: of artists, a drawback is that the solution of the diffusion process typically requires a global approach. Moreover, tra- ⎛ ⎞ ditional diffusion curves do not allow much control over the ⎜ ⎟ −33 0 0 colour propagation. Various methods that mitigate the latter ⎜ ⎟ Q = P = XP . (3) ⎝ ⎠ 00 −33 issue, such as diffusion barriers, anisotropic diffusion and colour strength, have been proposed in [4]. 123 952 P. J. Barendrecht et al. S(u, v) t t t t Fig. 4 Different colour blending functions with cubic parameterisation. Only the first one is G -compatible Likewise, P can be obtained from Q by inverting the Fig. 5 A Ferguson surface patch is defined by four control points and matrix X. eight tangent vectors. The visualised tangent vectors are scaled down by a factor 3 3.2 Colour blending and not in the interior, a natural choice is to use some kind of transfinite interpolation. More specifically, we consider Once the 2D geometry is defined, a colour can be assigned Coons patches that use a full set of cubics to blend the data to each Hermite control point in the curve network. Immedi- on the boundary. As the boundary curves are also cubics, ately, the question arises how the colour should be propagated this results in a bicubic patch S(u,v), which we choose to not only along the curves, but also within the quadrilateral describe in Hermite form as regions bounded by these curves. Assigning an RGB colour to each Hermite control point T S(u,v) = H (u)CH (v), (4) can be interpreted as extending the planar curve network to 5D. Alternatively, we can consider a separate height-field for with each colour channel on top of the 2D geometry, such that the ⎛ ⎞ S(0, 0) S (0, 0) S (0, 1) S(0, 1) v v z-component takes on the R, G or B value, which we choose ⎜ ⎟ S (0, 0) S (0, 0) S (0, 1) S (0, 1) u uv uv u to map to the interval [0, 1]. We use this latter view, extending ⎜ ⎟ C = , ⎝ ⎠ S (1, 0) S (1, 0) S (1, 1) S (1, 1) u uv uv u the curve network to 3D for each colour channel in turn. S(1, 0) S (1, 0) S (1, 1) S(1, 1) v v Following the assignment of z-values to the Hermite con- trol points,the z-values of the Hermite tangents are initialised where S(i , j ), S (i , j ), S (i , j ) and S (i , j ) are elements u v uv to 0. Assuming co-linear geometrical handles on the sides of of R with x and y the geometrical data and z the data from control points (see e.g., Fig. 8), this connects the curves in the one of the colour channels. 3D geometry/colour channel space with tangent continuity Initially, the mixed partials S (i , j ), with i , j ∈{0, 1} uv 1 1 (G ), and even C in case these geometrical handles are of (commonly referred to as twist vectors in Hermite form) at equal length. Clearly, other options for blending the colours the control points, are set to zero as we do not have inte- along the curves become available when the Hermite tan- rior information. This results in a subset of bicubic Hermite gents are assigned nonzero z-values, as illustrated in Fig. 4. patches known as Ferguson patches, illustrated in Fig. 5. Although this variation facilitates the use of different artis- When we cast the patch into Bézier form as tic styles, the drawback is that the colour curves no longer T −1 −T connect with tangent continuity. S(u,v) = B (u)X CX B(v), Whether tangent continuity is required or not depends on the interpretation of colour continuity, which is clearly dif- where we have used the equality ferent from geometric continuity as humans perceive spatial T T −1 T −T changes in a different way than changes in colour. This is Q H (t ) = P B(t ) = X Q B(t ) = Q X B(t ), most likely caused by the different spectral sensitivities of the three types of cone cells in the eyes [23], and might be it becomes evident that vanishing twist vectors correspond influenced by genetic and even cultural aspects. We postu- to inner Bézier points positioned in such a way that they form late that although tangent continuity might not be a necessary parallelograms with the closest corner and two corresponding condition for smooth colour propagation, it is a sufficient con- handles. dition. Therefore, we choose to use colour blending which When editing the geometry or colour and refining (indi- results in at least G continuity (see Fig. 4,far left). vidual) patches (see Sect. 4), the twist vectors might become Having fixed the colour blending along the curves, we nonzero, which is why we adopt the full Hermite description now take a look at the propagation of colour in the regions in favour of merely the Ferguson one in the remainder of this bounded by these curves. As we only have data on the curves paper. z (red channel) Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 953 1 1 1 1 2 8 2 8 3 1 3 1 − − − 2 4 2 4 Mesh Our implementation 1 1 1 1 2 8 2 8 3 1 3 1 − − − 2 4 2 4 Adobe Illustrator CorelDRAW Fig. 7 Four stencils to compute the quantities in (5)for u = 0or 1 and v = in Hermite form (green). Tangents are visualised as solid arrows, twist vectors as dashed arrows Inkscape (‘Bicubic’) Inkscape (‘Coons’) Fig. 6 Comparing renderings of a 2 × 3 gradient mesh in different soft- ware packages. Handles are at their default location and are visualised as small open circles We conclude this subsection with the observation that existing implementations of the traditional gradient mesh primitive use different blending functions, see Fig. 6. Used versions are Adobe Illustrator 2017.1.0, CorelDRAW Fig. 8 Splitting a patch using the stencils from Fig. 7 to compute posi- 19.0.0.328 (Graphics Suite 2017), and Inkscape 0.92.2. tions and tangent vectors. The twist vectors are not shown Based on these and other results, our approach resembles most that of CorelDRAW. Adobe Illustrator supposedly uses a different way of interpolation [25], which is also used in It follows that the unknown quantities in (5) can be com- the bicubic option in Inkscape. The other option in Inkscape puted just from the information available on the boundary of uses bilinear Coons interpolation. the patches. Using the matrix C introduced in (4), this results in the stencils shown in Fig. 7. 4 Refinement At this point, patches can be split by applying the stencils on two opposite curves, see Fig. 8. For example, a new control In our setting, (local) refinement (also referred to as splitting) 1 point at v = is obtained as a weighted sum (in this case refers to the introduction of a new curve that is the image of using the weights from the top stencil in Fig. 7)ofthe two an u-or v-isoline of the patch to be split. In the following, we control points and two tangent handles defining the curve of describe the splitting process along the u-direction, though interest. Examples of refinement are shown in Figs. 2, 14, 15, the process is completely analogous along the v-direction. 16 and 17, where individual patches have been split to locally Creating a new curve (expressed in Hermite form) involves accommodate more detail. Note that local refinement leads the evaluation of position and first derivatives (tangents). to T-sections in the mesh. In Sect. 5 we take a look at these Moreover, we need the mixed partial derivatives (twist vec- from the user’s point of view, whereas details regarding the tors) at the new control points to properly describe the new implementation and book-keeping concerning T-sections are patches. The required computations are based on the follow- discussed in Sect. 6. ing set of equations: We underline that our splitting approach is mathemati- ∂ S T T cally exact, which means that no change in geometry or S(u,v) = H (u)CH (v), = H (u)CH (v), ∂v colour propagation occurs upon splitting. The only difference ∂ S ∂ S could be caused by the (adaptive) tessellation step when it is T T = H (u)CH (v), = H (u)CH (v). (5) ∂u ∂u ∂v not pixel-accurate (see Sect. 6.2). In contrast, splitting meth- 123 954 P. J. Barendrecht et al. ods in existing implementations are rarely exact. Figure 9 compares the accuracy of the splits in the aforementioned software packages. The renderings before and after splitting have been imported in GIMP as different layers using 8 bits per colour channel, with the mode of the top layer set to differ- ence. The layers are subsequently merged and thresholded, Original mesh Vertical split Cross-split where the highest value resulting in white pixels has been recorded. Lower values for thresholding have been selected to emphasise the regions with the largest difference. Clearly, the splitting algorithms in Adobe Illustrator and CorelDRAW are not exact. In the former case, the twist vec- tors do not seem to be updated correctly, and in the latter case there is an issue most likely related to the computa- Our implementation 1 1 1 1 tion of tangent handles. Splitting in the case of Inkscape’s bicubic approach indicates severe problems regarding the computation of both position and tangent handles and pos- sibly twist vectors. The splitting procedure for Inkscape’s bilinear Coons implementation seems theoretically sound, though results in minor artefacts caused by the rendering approach. As mentioned, our implementation only shows a Adobe Illustrator 4 3 3 2 difference when the triangles created in the tessellation step are significantly larger than single pixels, and even in that case the difference is minimal. 5 User interaction CorelDRAW 7 4 11 6 In this section we discuss how the continuity of the colour surface is affected by user interaction. As an initial mesh (with or without local splits) is C by construction, we first look at what it takes to maintain this continuity. As it turns out that the resulting constraints can be somewhat restrictive, we subsequently discuss how the interaction can be made more flexible. Inkscape (‘Bicubic’) 43 26 68 32 5.1 C continuity The requirement for C continuity between two surface patches (mapped from the same domain) is straightforward [10]. In our bicubic setting, we have two sets of tangent han- dles and two sets of twist vectors that should remain co-linear Inkscape (‘Coons’) 1 1 2 1 and symmetric with respect to the shared curve. Of these, only Fig. 9 Comparing refinement accuracy. The left column shows the ren- the geometrical tangents can be edited by the user. Mirror- dering of an elementary gradient mesh (consisting of a single patch), ing the movement of one handle to the one opposite of it is the middle and right columns the differences after refining vertically sufficient to preserve local C continuity. Moving a tangent and in both directions (cross), respectively. The two values below each result are thresholding values (minimal value and selected value for vector also updates one or more inner Bézier points, as these better visual comparison) are related to the twist vectors which are kept constant. Note that nonzero twists might appear after movement of the tan- gent(s) logically orthogonal to a curve followed by (local) splitting, as is evident from the stencils in Fig. 7. Finally, This completes maintaining C in regular regions, and as is customary in vector graphics software, tangent handles leaves us to consider the same task around T-sections. In follow the movement of control points. this setting, data at T-sections cannot be modified by the 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 955 T T 1 2 P P 1 2 1 0 Fig. 11 User controls for the C setting (left) and general (C ) case (right), zooming in on the top-left part of the locally refined mesh in Fig. 10 A mesh with T-sections. Half-edges are indicated in grey. Green Fig. 1, here edited. Only the green colour channel is visualised, which arrows indicate the opposite of a half-edge, which is not always a sym- is colour-banded in order to assess continuity. In the general setting, metrical relation near T-sections. T-sections have fixed links to the tangent handles at the T-sections can be edited by the user, and tangent outgoing half-edge pointing in the direction of their stem, which are handles at control points no longer need to be (pairwise) co-linear, here indicated in dark red resulting in C transitions The last aspect to consider is an update of the colour of user. Instead, the T-sections inherit their information from a control point that is part of a curve containing T-sections. the longest curve (and associated patch) they are a part of. The procedure followed is analogous to the one for geom- Moving a geometrical control point or tangent handle influ- etry, including the occurrence of chain reactions which are encing a curve containing one or more T-sections triggers a required to correctly update the information at T-sections procedure which updates this data. In the case of position 1 further away from the affected curve in order to maintain C and tangent vectors along the long curve, this is straight- continuity. forward, whereas updating tangents in the other parametric direction and the twist vectors requires more effort because 5.2 More flexibility data from the adjacent patch need to be reflected and scaled by the appropriate factor. Note that modification of data at Forcing co-linear tangent vectors to be of the same length a T-section might again affect other T-sections. This only is required for C continuity, but might pose a restriction on happens when the stem (vertical part of an upright T)ofan the design of a desired image. Unfortunately, merely keeping updated T-section points at the bar (horizontal part of an them co-linear does not automatically imply G continu- upright T) of another T-section. For example, consider the ity. The requirements for this type of continuity are more mesh in Fig. 10, in which an update of the curve between P involved [9], and are at this point not discussed in more detail. and P triggers the update of the two T-sections T and T 2 1 2 Letting go of tangent continuity altogether exposes more on that curve. The stem of T points at the bar of T below 1 3 freedom to the user. In this setting, geometric tangent han- it, whose stem in turn points to the bar of T . dles can be edited individually, allowing the design of sharp Finally, we also update logically orthogonal tangent han- interior features. Tangent handles along the bar of a T-section dles at T-sections (as well as adjacent twist vectors) when are still updated automatically, but those in the other para- logically parallel tangent handles at control points on the metric direction (along the stem of a T-section) can now be curve containing the T-section are updated. For example, con- modified by the user. Figure 11 compares the user controls sider the tangent handle pointing down from T in Fig. 10. 1 0 for the C and general C setting. According to the stencils in Fig. 7 this handle (as well as its adjacent twist vectors) depends on the tangent handles pointing down from P and P . 1 2 Note that a change in geometry causes a change in colour 6 Implementation propagation because colour is defined on top of the geome- try. However, the actual colour values remain unchanged. Of In this section, we consider the relevant aspects of the imple- these values, only the colour at Hermite control points can be mentation, starting with an outline of the developed data updated by the user. The colour tangents directly follow from structure, followed by an overview of the employed adap- the choice of blending function along the curve (see Fig. 4) tive GPU-based tessellation. User controls such as moving and can only be nonzero following a (local) split, automati- points or handles, or colour picking from a background tex- cally maintaining a smooth connection. Colour twists follow ture, are not discussed. We refer to the supplementary video the colour tangents. for more details. 123 956 P. J. Barendrecht et al. 6.1 Data structure P P 1 2 Although local refinement for gradient meshes could be implemented using, e.g., (a forest of) binary trees, we instead 1 1 chose to use an augmented half-edge data structure to support level 0 2 2 additional and future extensions. Using a mixture of Bézier and Hermite representations, the 1 1 level 1 2 4 Hermite control points are connected by half-edges, which in turn link to the handles (stored in Bézier format). Each half- level 2 edge also links to a twist vector, stored in Hermite format. In Fig. 12 Sharing handles between pairs of half-edges of different levels. addition, colour information is also stored in the half-edges. The original handles are re-used and scaled by a factor after the first Near T-sections, the opposite (often referred to as twin or refinement, also resulting in two new handles on the sides of the T- pair) of an older half-edge is always the newer half-edge section. One of these is re-used and scaled after the second refinement, pointing to the origin of the older one. This ensures a stable whereas the other re-used handle is an original one scaled by a factor cycle around non-T-sections using the next and opposite links of a half-edge. Multiple newer half-edges can therefore have the same older half-edge as their opposite.The levelofa half-edge is increased by 1 every time it is split, where level 0 refers to the original (oldest) half-edges. Control points contain a flag indicating whether they are a T-section or not. At T-sections, the outgoing link from a control point to a half-edge is always the half-edge logically orthogonal to the T-section, whereas for other control points it can be an arbitrary outgoing half-edge like in the classi- cal half-edge structure. The reason for this fixed link is that it facilitates the traversal of an edge in both directions. Fig- Fig. 13 Adaptive tessellation showing compatible even integer tessel- ure 10 shows an example mesh with T-sections in which the lation levels as well as fractional levels behaviour deviating from a classical half-edge mesh is high- lighted. In the case of T-sections, there are two choices for the with the newest half-edge) of the curve of interest, a tessella- storage of handles along the adjacent pairs of half-edges of tion level is computed in a pre-processing step, which is then different level. One option would be to simply store two han- multiplied by the appropriate power of 2 (based on the differ- dles per half-edge. A more elegant solution is to share handles ence Δ in level of the two sides of the curve) for the longer between a pair of half-edges with different levels, which in sections of the edge. To illustrate this procedure, consider the practice amounts to the multiplication of the tangent handle short curve between T and T in Fig. 10. Assuming a com- 1 2 (in Hermite form) by a scaling factor which readily follows puted tessellation level of 6 for this curve, the curve between Δ=2 using de Casteljau’s algorithm. The result is that a T-section P and P is assigned a tessellation level of 6 · 2 = 24, 1 2 Δ=1 always has its own handles on both sides, whereas other han- likewise the curve between P and T level 6 · 2 = 12, 1 1 Δ=0 dles can be scaled versions of older ones. Figure 12 illustrates and finally the curve between T and P level 6 · 2 = 6. 2 2 this approach for the central edge P P in Fig. 10 where Currently, the common hardware limit for tessellation lev- 1 2 dyadic refinement has been applied. els is 64. In case of large patches or repeated refinement along a curve, the preferred level could exceed the maximum. In 6.2 Adaptive tessellation that case, the tessellation procedure can be extended such that these patches are virtually subdivided (subdivided only With the advent of programmable shaders in the graphics in memory), thereby mimicking levels of 128 and up. pipeline, also the rendering of vector graphics can be accel- For opposite half-edges of the same level, such as those erated [3]. In our application, the patches are rendered using between T and T in Fig. 10, the tessellation level is deter- 2 4 the OpenGL tessellation shaders in fractional even mode, mined in the tessellation control shader and can be fractional. which is mostly straightforward. Only when the levels of An example of the entire procedure is illustrated in Fig. 13. two opposite half-edges are different, special care needs to A more involved alternative would be to use the pixel- be taken to select an even integer tessellation level along that accurate approach from [13], although we should mention edge such that no cracks will occur. Using an approximation that this only considers accuracy in geometry, which does of the on-screen length of the shortest section (associated not automatically guarantee accuracy in colour. 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 957 Fig. 14 A plum modelled using local refinement and branching 7 Artistic features Fig. 15 The control net of a radial gradient mesh with local refinement, branches and multiple colours per control point enabling sharp colour The two contributions considered in this section are mostly transitions. The control points are enlarged for visualisation purposes interesting from an artistic point of view, as they bring more expressiveness to the primitive without introducing new tech- nical details. See the supplementary video for additional examples. 7.1 Branching Although the use of multiple, possibly (partially) overlapping Fig. 16 A wheel of cheese modelled using local refinement and sharp gradient meshes might still be the preferred way of designing colour transitions a complex shape, creating so-called branches in the mesh has proven to be an interesting alternative. Clearly, as we are no longer bound to a fixed rectangular topology, this allows for the design of meshes of more complex shape. The C conditions (see Sect. 5) can be enforced by reflecting tangents and twist vectors. Moreover, multiple branches can easily be stitched together to form closed loops (and therefore holes), a feature that can also be used to design genuine radial gradient meshes (see Fig. 15). In existing implementations, radial gradient meshes Fig. 17 Two husks of Physalis alkekengi (‘Chinese lantern’) modelled are merely rectangular gradient meshes with multiple over- using local refinement, branching and sharp colour transitions lapping control points, making them troublesome to edit. Figures 2, 14 and 17 show examples of gradient meshes with branching. different colours can be assigned to one control point. The control net shown in Fig. 15 contains various control points with multiple colours assigned to them. Examples of this 7.2 Sharp colour transitions extension are illustrated in Figs. 2, 16 and 17. In Sect. 6 we mentioned that colour information is stored in half-edges. The reason to do so is that it facilitates discon- tinuity of colour around a control point, which can be used 8 Discussion and conclusion to model sharp colour transitions similar to diffusion curves. This obviously violates continuity properties discussed in In this paper we have described our interpretation of a gradi- Sect. 5, but in return provides the artist with more freedom. ent mesh primitive extended with mathematically exact local As images often contain colour discontinuities, it is a very refinement, branching capabilities and sharp colour transi- useful feature to have. tions. Assigning the different colours is realised by only updat- Local refinement allows the user to add detail only where ing the colour of a specific sector of a control point. Each needed, such as local reflections of a light source. It helps the adjacent patch has its own sector, which means that up to 4 designer to maintain an overview of the topology of the mesh, 123 958 P. J. Barendrecht et al. Table 1 Comparing the number of patches required to model the exam- is available [18], it comes with several drawbacks as men- ples shown in this paper (figure numbers are indicated between brackets) tioned in Sect. 2. An alternative method based on polynomial using our version (local refinement and branching) and traditional gra- or rational patches, instead of the special patches required dient meshes (global refinement and possibly using multiple meshes for subdivision surfaces, could work. However, as tangent per object) continuity of the colour surface (in each channel) might be Leek (2) Plum (14) required, this is not a trivial task. Secondly, improved colour control would add to the range Ours 59 154 of styles supported by the primitive. Combining this with a Traditional 93 274 G blending function hints at introducing additional degrees Cheese (16) Lanterns (17) of freedom along individual curves. Ours 69 76 Combining both topics with vectorisation of raster graph- Traditional 126 145 ics would be a very challenging direction of research. Besides the artistic applications, it might also be interesting from the perspective of (image) compression, certainly in conjunction with local refinement. instead of getting lost in a clutter of points and curves which With the upcoming support of SVG 2 in most web might slow down the workflow either directly or indirectly. browsers, animated gradient meshes using, e.g., the data Table 1 compares global and local refinement by listing the structure introduced in [8] would make an interesting topic number of patches required to model the examples shown in for research. this paper; see also Fig. 1. Finally, as mentioned in [2], thenextstepinthisfield Branching takes away some of the topological restrictions could very well be the development of a primitive combining of the traditional gradient mesh, enabling the design of more the positive characteristics from both gradient meshes and complex objects using a single gradient mesh without reduc- diffusion curves. ing editability. The possibility to assign multiple colours to a single con- Acknowledgements Parts of this paper are based on, or were inspired trol point provides the artist with a user-friendly way to by, the BSc theses of the second and third authors. The real-life examples of gradient meshes were inspired by existing photographs (from third include sharp colour transitions in a design, one of the attrac- parties) under a Creative Commons license or similar and were created tive characteristics of diffusion curves. manually by the first author. Using adaptive crack-free tessellation allows for real-time editing, which remains interactive even for complex illustra- tions and (near) pixel-accuracy on our setup (i.e., an NVIDIA Compliance with ethical standards Quadro K420 GPU at a resolution of 2560 × 1440 pixels). Ultimately, the result is an efficient, versatile and expres- Conflict of interest The authors declare that they have no conflict of sive primitive, which is compatible with the PDF and new interest. SVG 2 standards. This compatibility makes it widely appli- Open Access This article is distributed under the terms of the Creative cable, in contrast to several existing methods, including Commons Attribution 4.0 International License (http://creativecomm subdivision-based ones. ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, Experience and user feedback with regard to editing has and reproduction in any medium, provided you give appropriate credit shown that certain topologies are difficult or even impossible to the original author(s) and the source, provide a link to the Creative to create using a single extended gradient mesh. Using mul- Commons license, and indicate if changes were made. tiple objects is currently the only available alternative (see Sect. 9 for future work on this aspect). In addition, certain aspects of illustrations would benefit from the availability of A Colour spaces other vector primitives. Integration of our tool in a general vector graphics editor is therefore desirable. In addition to the colour blending functions discussed in Sect. 3, we considered the use of different colour spaces. We briefly discuss the main characteristics of HSV, HSL, 9 Future work HSI, CIELCh (cylindrical spaces), and CIEXYZ, CIELUV, ab CIELAB, sRGB as well as linear sRGB. We conclude the paper with some promising ongoing and Cylindrical colour spaces represent hue in a single channel future work topics. First, more flexibility in topology of the H . Because of the periodical nature of H , interpolation can curve network is highly desirable. Although an approach be performed either clockwise or counter-clockwise, which based on subdivision surfaces for quad-dominant meshes turns out to be problematic. For example, interpolation in 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 959 References 1. Adobe Systems. Adobe Illustrator 8.0 Classroom in a Book. Adobe Press (1998) 2. Barla, P., Bousseau, A.: Gradient art: creation and vectorization. In: Rosin, P., Collomosse, J. (eds.) Image and Video-Based Artistic Stylisation, pp. 149–166. Springer, New York (2013) 3. Batra, V., Kilgard, M.J., Kumar, H., Lorach, T.: Accelerating vec- CIEXYZ CIELUV tor graphics rendering using the graphics hardware pipeline. ACM Trans. Graph. 34(4), 146 (2015) 4. Bezerra, H., Eisemann, E., DeCarlo, D., Thollot, J.: Diffusion con- straints for vector graphics. In: Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, pp. 35–42. ACM (2010) 5. Bowers, J.C., Leahey, J., Wang, R.: A ray tracing approach to diffu- sion curves. In: Computer Graphics Forum, vol. 30, pp. 1345–1352. Wiley (2011) CIELAB sRGB 6. Boyé, S., Barla, P., Guennebaud, G.: A vectorial solver for free- form vector gradients. ACM Trans. Graph. 31(6), 173 (2012) Fig. 18 Renderings of a 2 × 3 gradient mesh interpolated in different 7. Coburn, F.D., McCormick, P.: CorelDRAW 9: The Official Guide. colour spaces using G colour blending Osborne/McGraw-Hill, Berkeley (1999) 8. Dalstein, B., Ronfard, R., Van de Panne, M.: Vector graphics ani- mation with time-varying topology. ACM Trans. Graph. 34(4), 145 (2015) 9. DeRose, T.D.: Necessary and sufficient conditions for tangent plane ◦ ◦ HSV between red (H = 0 ) and magenta (H = 300 ) yields continuity of Bézier surfaces. Comput. Aided Geom. Des. 7(1–4), an undesired colour gradient including yellow, green, cyan 165–179 (1990) 10. Farin, G.E.: Curves and surfaces for CAGD: a practical guide. Mor- and blue when going counter-clockwise. Clearly, clockwise gan Kaufmann, Burlington (2002) interpolation is preferred in this case. Ambiguity of direction 11. Finch, M., Snyder, J., Hoppe, H.: Freeform vector graphics with occurs with a 180 difference between two colours, whereas controlled thin-plate splines. In: ACM Trans. Graph., vol. 30, p. subsequently tweaking one of these colours results in unsta- 166. ACM (2011) 12. Forsey, D.R., Bartels, R.H.: Hierarchical B-spline refinement. ble and non-intuitive behaviour. An alternative would be to ACM Siggraph Comput. Graph. 22(4), 205–212 (1988) linearly interpolate within the hue/saturation disc in HSV, 13. Hjelmervik, J.M., Fuchs, F.G.: Interactive pixel-accurate rendering though also in this case the results are often undesirable. We of LR-splines and T-splines. In: Eurographics (Short Papers), pp. conclude that cylindrical colour spaces are not suitable in the 65–68 (2015) 14. Inkscape Wiki. Mesh gradients (2018). http://wiki.inkscape.org/ context of gradient meshes. wiki/index.php/Mesh_Gradients. Accessed 01 Feb 2018 Results of using the other colour spaces are shown in 15. Lai, Y.-K., Hu, S.-M., Martin, R.R.: Automatic and topology- Fig. 18. Note that this overview does not include linear sRGB, preserving gradient mesh generation for image vectorization. In: as it is a linear transformation of CIEXYZ and therefore ACM Trans. Graph., vol. 28, p. 85. ACM (2009) 16. Li, X.-Y., Ju, T., Hu, S.-M.: Cubic mean value coordinates. ACM produces the same interpolation results. CIELUV, CIELAB Trans. Graph. 32(4), 126:1–126:10 (2013) and sRGB produce slightly darker results than CIEXYZ as 17. Liao, Z., Hoppe, H., Forsyth, D., Yu, Y.: A subdivision-based rep- they include a nonlinear gamma correction. Dark regions resentation for vector image editing. IEEE Trans. Vis. Comput. Graph. 18(11), 1858–1867 (2012) can occur when interpolating in sRGB between markedly 18. Lieng, H., Kosinka, J., Shen, J., Dodgson, N.A.: A colour interpo- different hues. Both CIELUV and CIELAB seem to be suit- lation scheme for topologically unrestricted gradient meshes. In: able spaces for colour interpolation, which makes sense as Computer Graphics Forum, vol. 36, pp. 112–121. Wiley (2017) they were specifically designed to be perceptually uniform. 19. Lieng, H., Tasse, F., Kosinka, J., Dodgson, N.A.: Shading curves: Of these two, CIELAB introduces some undesirable pur- vector-based drawing with explicit gradient control. In: Computer Graphics Forum, vol. 34, pp. 228–239. Wiley (2015) ple when interpolating from blue to yellow, which makes 20. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., CIELUV the best choice among the colour spaces consid- Salesin, D.: Diffusion curves: a vector representation for smooth- ered. shaded images. ACM Trans. Graph. 27(3), 92:1–92:8 (2008) In practice, the difference between rendering in the default 21. PDF Association. ISO 32000-2 (PDF 2.0). Technical report, Inter- national Organization for Standardization, (2017) sRGB colour space and CIELUV is hardly perceptible in our 22. Price, B., Barrett, W.: Object-based vectorization for interactive experience. The reason is that the difference in colour of image editing. Vis. Comput. 22(9), 661–670 (2006) adjacent control points is usually much less extreme than 23. Reinhard, E., Khan, E.A., Akyuz, A.O., Johnson, G.: Color imag- depicted in Fig. 18, and moreover, the patches are typically ing: fundamentals and applications. CRC Press, Boca Raton (2008) 24. Richardt, C., Lopez-Moreno, J., Bousseau, A., Agrawala, M., Dret- much smaller. In addition, conversion between sRGB and takis, G.: Vectorising bitmaps into semi-transparent gradient layers. CIELUV (on the GPU) is non-trivial. For these reasons, our In: Computer Graphics Forum, vol. 33, pp. 11–19. Wiley (2014) current implementation uses sRGB. 123 960 P. J. Barendrecht et al. Jonathan Hogervorst received his 25. Sun, J., Liang, L., Wen, F., Shum, H.-Y.: Image vectorization using B.Sc degrees in Artificial Intel- optimized gradient meshes. In: ACM Trans. Graph., vol. 26, p. 11. ligence and Computing Science ACM (2007) from the University of Gronin- 26. Sun, X., Xie, G., Dong, Y., Lin, S., Xu, W., Wang, W., Tong, X., gen, the Netherlands in 2017. He Guo, B.: Diffusion curve textures for resolution independent texture plans on following the M.Sc in mapping. ACM Trans. Graph. 31(4), 74 (2012) Security & Network Engineering 27. W3C SVG Working Group. Scalable Vector Graphics (SVG) 2 at the University of Amsterdam, (2018). https://www.w3.org/TR/SVG2/. Accessed 01 Feb 2018 the Netherlands. 28. Xia, T., Liao, B., Yu, Y.: Patch-based image vectorization with automatic curvilinear feature alignment. In: ACM Trans. Graph., vol. 28, p. 115. ACM (2009) 29. Xiao, Y., Wan, L., Leung, C.-S., Lai, Y.-K., Wong, T.-T.: Example- based color transfer for gradient meshes. IEEE Trans. Multimed. 15(3), 549–560 (2013) 30. Xiao, Y., Wan, L., Leung, C.S., Lai, Y.-K., Wong, T.-T.: Optimization-based gradient mesh colour transfer. In: Computer Graphics Forum, vol. 34, pp. 123–134. Wiley (2015) Jirí ˇ Kosinka is an Assistant Profes- 31. Zhou, H., Zheng, J., Wei, L.: Representing images using curvilinear sor in the Scientific Visualization feature driven subdivision surfaces. IEEE Trans. Image Process. and Computer Graphics research 23(8), 3268–3280 (2014) group at the Johann Bernoulli Institute of the University of Groningen, the Netherlands. He received his Ph.D. degree from Pieter J. Barendrecht obtained Charles University in Prague in his B.Sc and MSc degrees in 2006. His research interests Mechanical Engineering from include computer graphics, geo- Eindhoven University of Technol- metric modelling, and computer- ogy, the Netherlands. He is cur- aided design. rently pursuing a Ph.D. in Com- puter Graphics at the University of Groningen, the Netherlands, focusing on the application of splines in vector graphics, numer- ical analysis and 3D modelling. Martijn Luinstra is a Comput- ing Science student at the Uni- versity of Groningen, the Nether- lands. He plans on attending the M.Sc in Computing Science at the University of Groningen, with focus on computer graphics and data science. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Visual Computer Springer Journals

Locally refinable gradient meshes supporting branching and sharp colour transitions

Free
12 pages

Loading next page...
 
/lp/springer_journal/locally-refinable-gradient-meshes-supporting-branching-and-sharp-EOvCZIG0LY
Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s)
Subject
Computer Science; Computer Graphics; Computer Science, general; Artificial Intelligence (incl. Robotics); Image Processing and Computer Vision
ISSN
0178-2789
eISSN
1432-2315
D.O.I.
10.1007/s00371-018-1547-1
Publisher site
See Article on Publisher Site

Abstract

We present a local refinement approach for gradient meshes, a primitive commonly used in the design of vector illustrations with complex colour propagation. Local refinement allows the artist to add more detail only in the regions where it is needed, as opposed to global refinement which often clutters the workspace with undesired detail and potentially slows down the workflow. Moreover, in contrast to existing implementations of gradient mesh refinement, our approach ensures mathematically exact refinement. Additionally, we introduce a branching feature that allows for a wider range of mesh topologies, as well as a feature that enables sharp colour transitions similar to diffusion curves, which turn the gradient mesh into a more versatile and expressive vector graphics primitive. Keywords Vector graphics · Gradient meshes · Colour interpolation · Local refinement 1 Introduction the way gradient meshes behave in these software packages (see also Fig. 6), the overall concept is the same. In this Vector graphics provide an interesting alternative to raster paper we use the term traditional gradient mesh to refer to graphics, as they are resolution-independent and in gen- the primitive as implemented in these suites. eral easier to edit because they are composed of objects Upon using the traditional gradient mesh, it becomes clear (often called primitives) as opposed to coloured pixels. The that it comes with several limitations. The main shortcoming number of different primitives is rather limited, with the gra- is its fixed rectangular topology, making it difficult to design dient mesh as one of the most complex ones. This primitive complex objects and impossible to perform local refinement. facilitates the creation of intricate colour propagation over As is well-known in Computer Graphics, the latter aspect regular quadrilateral meshes, contributing to the possibility is a desirable feature [12]. Global refinement (propagating of designing photorealistic illustrations in vector format. through the entire row/column) often adds control points to Gradient meshes were introduced by Adobe Illustrator [1] the mesh where they are not needed, thereby cluttering the as the Mesh object. CorelDRAW soon followed [7] with a workspace and potentially slowing down the workflow. The similar tool called Mesh Fill. More recently, the primitive was conceptual difference between global and local refinement is added to the proposal for SVG 2 [27] and is now available in depicted in Fig. 1. In addition, the traditional gradient mesh Inkscape [14] as well. Although there are subtle differences in allows little to no control over the propagation of colour along curves and (as a consequence) in the interior, thus limiting the range of artistic styles that can be expressed with it. Electronic supplementary material The online version of this article It is our aim to create a more versatile and expressive prim- (https://doi.org/10.1007/s00371-018-1547-1) contains supplementary material, which is available to authorized users. itive, in particular from the perspective of manual creation and editing. Our contributions in this direction are: B Pieter J. Barendrecht p.j.barendrecht@rug.nl – Mathematically exact local refinement, which adds con- Bernoulli Institute, University of Groningen, Nijenborgh 9, 9747 Groningen, The Netherlands trol points only where they are required 123 950 P. J. Barendrecht et al. Fig. 3 Local refinement using Catmull–Clark subdivision. Vertices in the control net (orange) are generally not interpolated. Local refinement affects the valency of faces, which results in different limit surfaces (patch boundaries in black) Fig. 1 Global refinement (left), where each refinement propagates through an entire row/column, is the only form of refinement avail- 2 Related work able in existing implementations of the gradient mesh primitive. We propose a local refinement approach (right) The traditional gradient mesh primitive is conceptually straightforward. As such, the existing literature focuses on extensions or applications of the tool. The prior work can be classified into three different categories, discussing either the topology of the gradient mesh, aspects related to colour, or vectorisation, the (semi-) automatic creation of vector graphics from raster graphics. We conclude with a concise overview of alternative techniques. Considering the matter of restricted topology, in related fields subdivision surfaces are often used to model meshes of arbitrary topology. The catch in using them for gradient meshes is that neither interpolating subdivision schemes nor Fig. 2 Using a single gradient mesh to model a leek (control net top, approximating ones can be directly applied in a generally rendering bottom). Local refinement is used to express local details, the branching feature is used to create the non-rectangular topology satisfactory way. Using the former often results in colours desired for modelling this shape, and sharp colour transitions are used outside the gamut (which leads to flat or uniformly coloured to express depth spots), whereas the latter typically results in washed-out colours. The approaches taken by [17,31] use Loop’s approx- imating subdivision scheme for editing and vectorisation. – Branching, which enables the creation of non-rectangular Consequently, in practice the triangular elements in the mesh meshes by allowing branches, loops and (as a conse- need to be quite small for the intended colours to be approx- quence) holes imated closely enough (they are not interpolated), which – Assignment of colours to individual sectors of control in turn might complicate the editing process. This issue points, which facilitates sharp colour transitions is mostly avoided in [18] by combining augmented lin- ear ternary subdivision and Catmull–Clark’s approximating scheme supporting quad-dominant meshes of arbitrary topol- An example gradient mesh using all three novel features ogy. However, though colour is interpolated, by default the is shown in Fig. 2. geometry (represented by the control net) is not. Moreover, The structure of the remainder of this paper is as fol- like [17,31], the proposed method does not allow for local lows. In Sect. 2, we discuss and compare related work on refinement without changing the geometry or the colour prop- gradient meshes and alternative tools. We describe our inter- agation. In this subdivision setting, local refinement changes pretation of the gradient mesh primitive in Sect. 3, focusing the valency of faces and/or introduces vertices of irregular on geometry and colour blending. Refinement is discussed valencies, thereby changing the limit surface. See Fig. 3,cf. in detail in Sect. 4. User interaction (i.e., editing a gradient Fig. 1 (right). mesh) is considered in Sect. 5. Relevant details regarding the Finally, neither the PDF 2.0 [21] nor the SVG 2 standard implementation are described in Sect. 6. Branching and sharp supports the surface patches required to render subdivision colour transitions are described in Sect. 7 and are illustrated surfaces, which reduces the applicability of subdivision- using a number of examples (see also the supplementary based methods. video). Finally, we evaluate the advantages and limitations Methods not based on subdivision include the use of of our contributions in Sect. 8, and look ahead to future work curved Bézier triangles [28] for vectorisation. The approach in Sect. 9. touches only briefly on colour continuity across two trian- 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 951 gular elements, which is not sufficient for overall continuity. Related to diffusion curves, [19] associates shading pro- Although editing of the generated elements is discussed to files to the curves and subsequently converts the scene to a some extent, the method does not appear to support manual Catmull–Clark subdivision surface which can be rendered addition of elements. Finally, [16] considers vectorisation as directly. As it is subdivision-based, it comes with the same an application of their generalised barycentric coordinates drawbacks as mentioned above. for Hermite interpolation, which allows the use of highly irregular patch shapes. While interesting for vectorisation, the method does not seem suitable for intuitive manual cre- 3 The gradient mesh primitive ation of gradient meshes. To the best of our knowledge, no extensive comparison We now construct our interpretation of the gradient mesh regarding the use of different colour spaces for gradient primitive. First, we consider the geometrical aspects, after meshes has been published. However, preference for cer- which we focus on blending the colours defined on top of the tain colour spaces, including CIELAB, has been mentioned 2D geometry. In addition, “Appendix A” briefly discusses the before [18]. Instead, work on colour-related aspects focuses use of different colour spaces. on colour transfer, which discusses how colour characteris- As pointed out in [24], (semi-)transparency in vector tics from an existing (raster) image can be used to (re-)colour graphics is an important feature. In addition to the R, G and a gradient mesh [29,30]. B channels, a fourth channel A could be added to smoothly Most work primarily focuses on vectorisation. In [22], blend different alpha values assigned to control points. cubic Bézier patches are used to approximate individual To fully appreciate this feature, a layer system should be objects extracted from raster images. Interestingly, it also integrated to handle overlapping meshes. Although straight- discusses the need for (in this case, automatic) local refine- forward to implement, our framework currently does not ment to efficiently express local detail. However, smoothness support it as it adds little value from a scientific point of of the result is not guaranteed, and locally refined areas can- view. In the following, we adopt the RGB colour space. not be edited directly. A different approach is taken in [25], where the positions and other properties at control points of a 3.1 Geometry given (manually created) gradient mesh of rectangular topol- ogy can be optimised. In contrast, [15] proposes a completely In essence, the geometrical aspect consists of a 2D curve net- automatic approach, which is more generally applicable as it work in the xy-plane with (traditionally) the topology of a can vectorise objects with holes using a single mesh. How- rectangular grid. An example is shown in Fig. 1 (left). The ever, like [25], it uses rectangular meshes, and holes can only curves composing this network are commonly cubic curves. be formed by separating surface patches along a chain of Throughout the paper we use both Bézier and Hermite formu- curve segments, which restricts their shape. lations of these curves (and later surface patches). As such, Obtaining the desired topology using a (traditional) gra- each cubic curve C (t ) can be described as dient mesh can be quite a challenge. A common workflow is to use multiple gradient meshes to design the envisioned 3 object. Still, manual creation of complex shapes remains C (t ) = P B (t ) = P B(t ) (1) k k time-consuming. Diffusion curves [20] alleviate this task by k=0 allowing the user to draw outlines and open or closed inner = Q H (t ) = Q H (t ), (2) regions where changes in colour might occur, using Bézier k k k=0 or B-spline curves. Assigning (possibly different) colours to the two sides of the curves, the interior colour is determined where t ∈[0, 1]. Note that P are the Bézier control points using the principle of diffusion. From a mathematical point of and B (t ) the cubic Bernstein functions, Q the Hermite con- k k view, this is interpreted as solving a harmonic or biharmonic trols (points or tangent handles) and H (t ) the cubic Hermite [11] equation. In addition to the multigrid method used orig- functions. inally, existing approaches for solving these PDEs include As two different representations are considered, a way to FEM [6], BEM [26] and approximation by ray-tracing [5]. convert between them is desirable. Hermite control data Q Although the workflow is closer to the classical workflow can readily be obtained from the Bézier control data P: of artists, a drawback is that the solution of the diffusion process typically requires a global approach. Moreover, tra- ⎛ ⎞ ditional diffusion curves do not allow much control over the ⎜ ⎟ −33 0 0 colour propagation. Various methods that mitigate the latter ⎜ ⎟ Q = P = XP . (3) ⎝ ⎠ 00 −33 issue, such as diffusion barriers, anisotropic diffusion and colour strength, have been proposed in [4]. 123 952 P. J. Barendrecht et al. S(u, v) t t t t Fig. 4 Different colour blending functions with cubic parameterisation. Only the first one is G -compatible Likewise, P can be obtained from Q by inverting the Fig. 5 A Ferguson surface patch is defined by four control points and matrix X. eight tangent vectors. The visualised tangent vectors are scaled down by a factor 3 3.2 Colour blending and not in the interior, a natural choice is to use some kind of transfinite interpolation. More specifically, we consider Once the 2D geometry is defined, a colour can be assigned Coons patches that use a full set of cubics to blend the data to each Hermite control point in the curve network. Immedi- on the boundary. As the boundary curves are also cubics, ately, the question arises how the colour should be propagated this results in a bicubic patch S(u,v), which we choose to not only along the curves, but also within the quadrilateral describe in Hermite form as regions bounded by these curves. Assigning an RGB colour to each Hermite control point T S(u,v) = H (u)CH (v), (4) can be interpreted as extending the planar curve network to 5D. Alternatively, we can consider a separate height-field for with each colour channel on top of the 2D geometry, such that the ⎛ ⎞ S(0, 0) S (0, 0) S (0, 1) S(0, 1) v v z-component takes on the R, G or B value, which we choose ⎜ ⎟ S (0, 0) S (0, 0) S (0, 1) S (0, 1) u uv uv u to map to the interval [0, 1]. We use this latter view, extending ⎜ ⎟ C = , ⎝ ⎠ S (1, 0) S (1, 0) S (1, 1) S (1, 1) u uv uv u the curve network to 3D for each colour channel in turn. S(1, 0) S (1, 0) S (1, 1) S(1, 1) v v Following the assignment of z-values to the Hermite con- trol points,the z-values of the Hermite tangents are initialised where S(i , j ), S (i , j ), S (i , j ) and S (i , j ) are elements u v uv to 0. Assuming co-linear geometrical handles on the sides of of R with x and y the geometrical data and z the data from control points (see e.g., Fig. 8), this connects the curves in the one of the colour channels. 3D geometry/colour channel space with tangent continuity Initially, the mixed partials S (i , j ), with i , j ∈{0, 1} uv 1 1 (G ), and even C in case these geometrical handles are of (commonly referred to as twist vectors in Hermite form) at equal length. Clearly, other options for blending the colours the control points, are set to zero as we do not have inte- along the curves become available when the Hermite tan- rior information. This results in a subset of bicubic Hermite gents are assigned nonzero z-values, as illustrated in Fig. 4. patches known as Ferguson patches, illustrated in Fig. 5. Although this variation facilitates the use of different artis- When we cast the patch into Bézier form as tic styles, the drawback is that the colour curves no longer T −1 −T connect with tangent continuity. S(u,v) = B (u)X CX B(v), Whether tangent continuity is required or not depends on the interpretation of colour continuity, which is clearly dif- where we have used the equality ferent from geometric continuity as humans perceive spatial T T −1 T −T changes in a different way than changes in colour. This is Q H (t ) = P B(t ) = X Q B(t ) = Q X B(t ), most likely caused by the different spectral sensitivities of the three types of cone cells in the eyes [23], and might be it becomes evident that vanishing twist vectors correspond influenced by genetic and even cultural aspects. We postu- to inner Bézier points positioned in such a way that they form late that although tangent continuity might not be a necessary parallelograms with the closest corner and two corresponding condition for smooth colour propagation, it is a sufficient con- handles. dition. Therefore, we choose to use colour blending which When editing the geometry or colour and refining (indi- results in at least G continuity (see Fig. 4,far left). vidual) patches (see Sect. 4), the twist vectors might become Having fixed the colour blending along the curves, we nonzero, which is why we adopt the full Hermite description now take a look at the propagation of colour in the regions in favour of merely the Ferguson one in the remainder of this bounded by these curves. As we only have data on the curves paper. z (red channel) Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 953 1 1 1 1 2 8 2 8 3 1 3 1 − − − 2 4 2 4 Mesh Our implementation 1 1 1 1 2 8 2 8 3 1 3 1 − − − 2 4 2 4 Adobe Illustrator CorelDRAW Fig. 7 Four stencils to compute the quantities in (5)for u = 0or 1 and v = in Hermite form (green). Tangents are visualised as solid arrows, twist vectors as dashed arrows Inkscape (‘Bicubic’) Inkscape (‘Coons’) Fig. 6 Comparing renderings of a 2 × 3 gradient mesh in different soft- ware packages. Handles are at their default location and are visualised as small open circles We conclude this subsection with the observation that existing implementations of the traditional gradient mesh primitive use different blending functions, see Fig. 6. Used versions are Adobe Illustrator 2017.1.0, CorelDRAW Fig. 8 Splitting a patch using the stencils from Fig. 7 to compute posi- 19.0.0.328 (Graphics Suite 2017), and Inkscape 0.92.2. tions and tangent vectors. The twist vectors are not shown Based on these and other results, our approach resembles most that of CorelDRAW. Adobe Illustrator supposedly uses a different way of interpolation [25], which is also used in It follows that the unknown quantities in (5) can be com- the bicubic option in Inkscape. The other option in Inkscape puted just from the information available on the boundary of uses bilinear Coons interpolation. the patches. Using the matrix C introduced in (4), this results in the stencils shown in Fig. 7. 4 Refinement At this point, patches can be split by applying the stencils on two opposite curves, see Fig. 8. For example, a new control In our setting, (local) refinement (also referred to as splitting) 1 point at v = is obtained as a weighted sum (in this case refers to the introduction of a new curve that is the image of using the weights from the top stencil in Fig. 7)ofthe two an u-or v-isoline of the patch to be split. In the following, we control points and two tangent handles defining the curve of describe the splitting process along the u-direction, though interest. Examples of refinement are shown in Figs. 2, 14, 15, the process is completely analogous along the v-direction. 16 and 17, where individual patches have been split to locally Creating a new curve (expressed in Hermite form) involves accommodate more detail. Note that local refinement leads the evaluation of position and first derivatives (tangents). to T-sections in the mesh. In Sect. 5 we take a look at these Moreover, we need the mixed partial derivatives (twist vec- from the user’s point of view, whereas details regarding the tors) at the new control points to properly describe the new implementation and book-keeping concerning T-sections are patches. The required computations are based on the follow- discussed in Sect. 6. ing set of equations: We underline that our splitting approach is mathemati- ∂ S T T cally exact, which means that no change in geometry or S(u,v) = H (u)CH (v), = H (u)CH (v), ∂v colour propagation occurs upon splitting. The only difference ∂ S ∂ S could be caused by the (adaptive) tessellation step when it is T T = H (u)CH (v), = H (u)CH (v). (5) ∂u ∂u ∂v not pixel-accurate (see Sect. 6.2). In contrast, splitting meth- 123 954 P. J. Barendrecht et al. ods in existing implementations are rarely exact. Figure 9 compares the accuracy of the splits in the aforementioned software packages. The renderings before and after splitting have been imported in GIMP as different layers using 8 bits per colour channel, with the mode of the top layer set to differ- ence. The layers are subsequently merged and thresholded, Original mesh Vertical split Cross-split where the highest value resulting in white pixels has been recorded. Lower values for thresholding have been selected to emphasise the regions with the largest difference. Clearly, the splitting algorithms in Adobe Illustrator and CorelDRAW are not exact. In the former case, the twist vec- tors do not seem to be updated correctly, and in the latter case there is an issue most likely related to the computa- Our implementation 1 1 1 1 tion of tangent handles. Splitting in the case of Inkscape’s bicubic approach indicates severe problems regarding the computation of both position and tangent handles and pos- sibly twist vectors. The splitting procedure for Inkscape’s bilinear Coons implementation seems theoretically sound, though results in minor artefacts caused by the rendering approach. As mentioned, our implementation only shows a Adobe Illustrator 4 3 3 2 difference when the triangles created in the tessellation step are significantly larger than single pixels, and even in that case the difference is minimal. 5 User interaction CorelDRAW 7 4 11 6 In this section we discuss how the continuity of the colour surface is affected by user interaction. As an initial mesh (with or without local splits) is C by construction, we first look at what it takes to maintain this continuity. As it turns out that the resulting constraints can be somewhat restrictive, we subsequently discuss how the interaction can be made more flexible. Inkscape (‘Bicubic’) 43 26 68 32 5.1 C continuity The requirement for C continuity between two surface patches (mapped from the same domain) is straightforward [10]. In our bicubic setting, we have two sets of tangent han- dles and two sets of twist vectors that should remain co-linear Inkscape (‘Coons’) 1 1 2 1 and symmetric with respect to the shared curve. Of these, only Fig. 9 Comparing refinement accuracy. The left column shows the ren- the geometrical tangents can be edited by the user. Mirror- dering of an elementary gradient mesh (consisting of a single patch), ing the movement of one handle to the one opposite of it is the middle and right columns the differences after refining vertically sufficient to preserve local C continuity. Moving a tangent and in both directions (cross), respectively. The two values below each result are thresholding values (minimal value and selected value for vector also updates one or more inner Bézier points, as these better visual comparison) are related to the twist vectors which are kept constant. Note that nonzero twists might appear after movement of the tan- gent(s) logically orthogonal to a curve followed by (local) splitting, as is evident from the stencils in Fig. 7. Finally, This completes maintaining C in regular regions, and as is customary in vector graphics software, tangent handles leaves us to consider the same task around T-sections. In follow the movement of control points. this setting, data at T-sections cannot be modified by the 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 955 T T 1 2 P P 1 2 1 0 Fig. 11 User controls for the C setting (left) and general (C ) case (right), zooming in on the top-left part of the locally refined mesh in Fig. 10 A mesh with T-sections. Half-edges are indicated in grey. Green Fig. 1, here edited. Only the green colour channel is visualised, which arrows indicate the opposite of a half-edge, which is not always a sym- is colour-banded in order to assess continuity. In the general setting, metrical relation near T-sections. T-sections have fixed links to the tangent handles at the T-sections can be edited by the user, and tangent outgoing half-edge pointing in the direction of their stem, which are handles at control points no longer need to be (pairwise) co-linear, here indicated in dark red resulting in C transitions The last aspect to consider is an update of the colour of user. Instead, the T-sections inherit their information from a control point that is part of a curve containing T-sections. the longest curve (and associated patch) they are a part of. The procedure followed is analogous to the one for geom- Moving a geometrical control point or tangent handle influ- etry, including the occurrence of chain reactions which are encing a curve containing one or more T-sections triggers a required to correctly update the information at T-sections procedure which updates this data. In the case of position 1 further away from the affected curve in order to maintain C and tangent vectors along the long curve, this is straight- continuity. forward, whereas updating tangents in the other parametric direction and the twist vectors requires more effort because 5.2 More flexibility data from the adjacent patch need to be reflected and scaled by the appropriate factor. Note that modification of data at Forcing co-linear tangent vectors to be of the same length a T-section might again affect other T-sections. This only is required for C continuity, but might pose a restriction on happens when the stem (vertical part of an upright T)ofan the design of a desired image. Unfortunately, merely keeping updated T-section points at the bar (horizontal part of an them co-linear does not automatically imply G continu- upright T) of another T-section. For example, consider the ity. The requirements for this type of continuity are more mesh in Fig. 10, in which an update of the curve between P involved [9], and are at this point not discussed in more detail. and P triggers the update of the two T-sections T and T 2 1 2 Letting go of tangent continuity altogether exposes more on that curve. The stem of T points at the bar of T below 1 3 freedom to the user. In this setting, geometric tangent han- it, whose stem in turn points to the bar of T . dles can be edited individually, allowing the design of sharp Finally, we also update logically orthogonal tangent han- interior features. Tangent handles along the bar of a T-section dles at T-sections (as well as adjacent twist vectors) when are still updated automatically, but those in the other para- logically parallel tangent handles at control points on the metric direction (along the stem of a T-section) can now be curve containing the T-section are updated. For example, con- modified by the user. Figure 11 compares the user controls sider the tangent handle pointing down from T in Fig. 10. 1 0 for the C and general C setting. According to the stencils in Fig. 7 this handle (as well as its adjacent twist vectors) depends on the tangent handles pointing down from P and P . 1 2 Note that a change in geometry causes a change in colour 6 Implementation propagation because colour is defined on top of the geome- try. However, the actual colour values remain unchanged. Of In this section, we consider the relevant aspects of the imple- these values, only the colour at Hermite control points can be mentation, starting with an outline of the developed data updated by the user. The colour tangents directly follow from structure, followed by an overview of the employed adap- the choice of blending function along the curve (see Fig. 4) tive GPU-based tessellation. User controls such as moving and can only be nonzero following a (local) split, automati- points or handles, or colour picking from a background tex- cally maintaining a smooth connection. Colour twists follow ture, are not discussed. We refer to the supplementary video the colour tangents. for more details. 123 956 P. J. Barendrecht et al. 6.1 Data structure P P 1 2 Although local refinement for gradient meshes could be implemented using, e.g., (a forest of) binary trees, we instead 1 1 chose to use an augmented half-edge data structure to support level 0 2 2 additional and future extensions. Using a mixture of Bézier and Hermite representations, the 1 1 level 1 2 4 Hermite control points are connected by half-edges, which in turn link to the handles (stored in Bézier format). Each half- level 2 edge also links to a twist vector, stored in Hermite format. In Fig. 12 Sharing handles between pairs of half-edges of different levels. addition, colour information is also stored in the half-edges. The original handles are re-used and scaled by a factor after the first Near T-sections, the opposite (often referred to as twin or refinement, also resulting in two new handles on the sides of the T- pair) of an older half-edge is always the newer half-edge section. One of these is re-used and scaled after the second refinement, pointing to the origin of the older one. This ensures a stable whereas the other re-used handle is an original one scaled by a factor cycle around non-T-sections using the next and opposite links of a half-edge. Multiple newer half-edges can therefore have the same older half-edge as their opposite.The levelofa half-edge is increased by 1 every time it is split, where level 0 refers to the original (oldest) half-edges. Control points contain a flag indicating whether they are a T-section or not. At T-sections, the outgoing link from a control point to a half-edge is always the half-edge logically orthogonal to the T-section, whereas for other control points it can be an arbitrary outgoing half-edge like in the classi- cal half-edge structure. The reason for this fixed link is that it facilitates the traversal of an edge in both directions. Fig- Fig. 13 Adaptive tessellation showing compatible even integer tessel- ure 10 shows an example mesh with T-sections in which the lation levels as well as fractional levels behaviour deviating from a classical half-edge mesh is high- lighted. In the case of T-sections, there are two choices for the with the newest half-edge) of the curve of interest, a tessella- storage of handles along the adjacent pairs of half-edges of tion level is computed in a pre-processing step, which is then different level. One option would be to simply store two han- multiplied by the appropriate power of 2 (based on the differ- dles per half-edge. A more elegant solution is to share handles ence Δ in level of the two sides of the curve) for the longer between a pair of half-edges with different levels, which in sections of the edge. To illustrate this procedure, consider the practice amounts to the multiplication of the tangent handle short curve between T and T in Fig. 10. Assuming a com- 1 2 (in Hermite form) by a scaling factor which readily follows puted tessellation level of 6 for this curve, the curve between Δ=2 using de Casteljau’s algorithm. The result is that a T-section P and P is assigned a tessellation level of 6 · 2 = 24, 1 2 Δ=1 always has its own handles on both sides, whereas other han- likewise the curve between P and T level 6 · 2 = 12, 1 1 Δ=0 dles can be scaled versions of older ones. Figure 12 illustrates and finally the curve between T and P level 6 · 2 = 6. 2 2 this approach for the central edge P P in Fig. 10 where Currently, the common hardware limit for tessellation lev- 1 2 dyadic refinement has been applied. els is 64. In case of large patches or repeated refinement along a curve, the preferred level could exceed the maximum. In 6.2 Adaptive tessellation that case, the tessellation procedure can be extended such that these patches are virtually subdivided (subdivided only With the advent of programmable shaders in the graphics in memory), thereby mimicking levels of 128 and up. pipeline, also the rendering of vector graphics can be accel- For opposite half-edges of the same level, such as those erated [3]. In our application, the patches are rendered using between T and T in Fig. 10, the tessellation level is deter- 2 4 the OpenGL tessellation shaders in fractional even mode, mined in the tessellation control shader and can be fractional. which is mostly straightforward. Only when the levels of An example of the entire procedure is illustrated in Fig. 13. two opposite half-edges are different, special care needs to A more involved alternative would be to use the pixel- be taken to select an even integer tessellation level along that accurate approach from [13], although we should mention edge such that no cracks will occur. Using an approximation that this only considers accuracy in geometry, which does of the on-screen length of the shortest section (associated not automatically guarantee accuracy in colour. 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 957 Fig. 14 A plum modelled using local refinement and branching 7 Artistic features Fig. 15 The control net of a radial gradient mesh with local refinement, branches and multiple colours per control point enabling sharp colour The two contributions considered in this section are mostly transitions. The control points are enlarged for visualisation purposes interesting from an artistic point of view, as they bring more expressiveness to the primitive without introducing new tech- nical details. See the supplementary video for additional examples. 7.1 Branching Although the use of multiple, possibly (partially) overlapping Fig. 16 A wheel of cheese modelled using local refinement and sharp gradient meshes might still be the preferred way of designing colour transitions a complex shape, creating so-called branches in the mesh has proven to be an interesting alternative. Clearly, as we are no longer bound to a fixed rectangular topology, this allows for the design of meshes of more complex shape. The C conditions (see Sect. 5) can be enforced by reflecting tangents and twist vectors. Moreover, multiple branches can easily be stitched together to form closed loops (and therefore holes), a feature that can also be used to design genuine radial gradient meshes (see Fig. 15). In existing implementations, radial gradient meshes Fig. 17 Two husks of Physalis alkekengi (‘Chinese lantern’) modelled are merely rectangular gradient meshes with multiple over- using local refinement, branching and sharp colour transitions lapping control points, making them troublesome to edit. Figures 2, 14 and 17 show examples of gradient meshes with branching. different colours can be assigned to one control point. The control net shown in Fig. 15 contains various control points with multiple colours assigned to them. Examples of this 7.2 Sharp colour transitions extension are illustrated in Figs. 2, 16 and 17. In Sect. 6 we mentioned that colour information is stored in half-edges. The reason to do so is that it facilitates discon- tinuity of colour around a control point, which can be used 8 Discussion and conclusion to model sharp colour transitions similar to diffusion curves. This obviously violates continuity properties discussed in In this paper we have described our interpretation of a gradi- Sect. 5, but in return provides the artist with more freedom. ent mesh primitive extended with mathematically exact local As images often contain colour discontinuities, it is a very refinement, branching capabilities and sharp colour transi- useful feature to have. tions. Assigning the different colours is realised by only updat- Local refinement allows the user to add detail only where ing the colour of a specific sector of a control point. Each needed, such as local reflections of a light source. It helps the adjacent patch has its own sector, which means that up to 4 designer to maintain an overview of the topology of the mesh, 123 958 P. J. Barendrecht et al. Table 1 Comparing the number of patches required to model the exam- is available [18], it comes with several drawbacks as men- ples shown in this paper (figure numbers are indicated between brackets) tioned in Sect. 2. An alternative method based on polynomial using our version (local refinement and branching) and traditional gra- or rational patches, instead of the special patches required dient meshes (global refinement and possibly using multiple meshes for subdivision surfaces, could work. However, as tangent per object) continuity of the colour surface (in each channel) might be Leek (2) Plum (14) required, this is not a trivial task. Secondly, improved colour control would add to the range Ours 59 154 of styles supported by the primitive. Combining this with a Traditional 93 274 G blending function hints at introducing additional degrees Cheese (16) Lanterns (17) of freedom along individual curves. Ours 69 76 Combining both topics with vectorisation of raster graph- Traditional 126 145 ics would be a very challenging direction of research. Besides the artistic applications, it might also be interesting from the perspective of (image) compression, certainly in conjunction with local refinement. instead of getting lost in a clutter of points and curves which With the upcoming support of SVG 2 in most web might slow down the workflow either directly or indirectly. browsers, animated gradient meshes using, e.g., the data Table 1 compares global and local refinement by listing the structure introduced in [8] would make an interesting topic number of patches required to model the examples shown in for research. this paper; see also Fig. 1. Finally, as mentioned in [2], thenextstepinthisfield Branching takes away some of the topological restrictions could very well be the development of a primitive combining of the traditional gradient mesh, enabling the design of more the positive characteristics from both gradient meshes and complex objects using a single gradient mesh without reduc- diffusion curves. ing editability. The possibility to assign multiple colours to a single con- Acknowledgements Parts of this paper are based on, or were inspired trol point provides the artist with a user-friendly way to by, the BSc theses of the second and third authors. The real-life examples of gradient meshes were inspired by existing photographs (from third include sharp colour transitions in a design, one of the attrac- parties) under a Creative Commons license or similar and were created tive characteristics of diffusion curves. manually by the first author. Using adaptive crack-free tessellation allows for real-time editing, which remains interactive even for complex illustra- tions and (near) pixel-accuracy on our setup (i.e., an NVIDIA Compliance with ethical standards Quadro K420 GPU at a resolution of 2560 × 1440 pixels). Ultimately, the result is an efficient, versatile and expres- Conflict of interest The authors declare that they have no conflict of sive primitive, which is compatible with the PDF and new interest. SVG 2 standards. This compatibility makes it widely appli- Open Access This article is distributed under the terms of the Creative cable, in contrast to several existing methods, including Commons Attribution 4.0 International License (http://creativecomm subdivision-based ones. ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, Experience and user feedback with regard to editing has and reproduction in any medium, provided you give appropriate credit shown that certain topologies are difficult or even impossible to the original author(s) and the source, provide a link to the Creative to create using a single extended gradient mesh. Using mul- Commons license, and indicate if changes were made. tiple objects is currently the only available alternative (see Sect. 9 for future work on this aspect). In addition, certain aspects of illustrations would benefit from the availability of A Colour spaces other vector primitives. Integration of our tool in a general vector graphics editor is therefore desirable. In addition to the colour blending functions discussed in Sect. 3, we considered the use of different colour spaces. We briefly discuss the main characteristics of HSV, HSL, 9 Future work HSI, CIELCh (cylindrical spaces), and CIEXYZ, CIELUV, ab CIELAB, sRGB as well as linear sRGB. We conclude the paper with some promising ongoing and Cylindrical colour spaces represent hue in a single channel future work topics. First, more flexibility in topology of the H . Because of the periodical nature of H , interpolation can curve network is highly desirable. Although an approach be performed either clockwise or counter-clockwise, which based on subdivision surfaces for quad-dominant meshes turns out to be problematic. For example, interpolation in 123 Locally refinable gradient meshes supporting branching and sharp colour transitions: towards… 959 References 1. Adobe Systems. Adobe Illustrator 8.0 Classroom in a Book. Adobe Press (1998) 2. Barla, P., Bousseau, A.: Gradient art: creation and vectorization. In: Rosin, P., Collomosse, J. (eds.) Image and Video-Based Artistic Stylisation, pp. 149–166. Springer, New York (2013) 3. Batra, V., Kilgard, M.J., Kumar, H., Lorach, T.: Accelerating vec- CIEXYZ CIELUV tor graphics rendering using the graphics hardware pipeline. ACM Trans. Graph. 34(4), 146 (2015) 4. Bezerra, H., Eisemann, E., DeCarlo, D., Thollot, J.: Diffusion con- straints for vector graphics. In: Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, pp. 35–42. ACM (2010) 5. Bowers, J.C., Leahey, J., Wang, R.: A ray tracing approach to diffu- sion curves. In: Computer Graphics Forum, vol. 30, pp. 1345–1352. Wiley (2011) CIELAB sRGB 6. Boyé, S., Barla, P., Guennebaud, G.: A vectorial solver for free- form vector gradients. ACM Trans. Graph. 31(6), 173 (2012) Fig. 18 Renderings of a 2 × 3 gradient mesh interpolated in different 7. Coburn, F.D., McCormick, P.: CorelDRAW 9: The Official Guide. colour spaces using G colour blending Osborne/McGraw-Hill, Berkeley (1999) 8. Dalstein, B., Ronfard, R., Van de Panne, M.: Vector graphics ani- mation with time-varying topology. ACM Trans. Graph. 34(4), 145 (2015) 9. DeRose, T.D.: Necessary and sufficient conditions for tangent plane ◦ ◦ HSV between red (H = 0 ) and magenta (H = 300 ) yields continuity of Bézier surfaces. Comput. Aided Geom. Des. 7(1–4), an undesired colour gradient including yellow, green, cyan 165–179 (1990) 10. Farin, G.E.: Curves and surfaces for CAGD: a practical guide. Mor- and blue when going counter-clockwise. Clearly, clockwise gan Kaufmann, Burlington (2002) interpolation is preferred in this case. Ambiguity of direction 11. Finch, M., Snyder, J., Hoppe, H.: Freeform vector graphics with occurs with a 180 difference between two colours, whereas controlled thin-plate splines. In: ACM Trans. Graph., vol. 30, p. subsequently tweaking one of these colours results in unsta- 166. ACM (2011) 12. Forsey, D.R., Bartels, R.H.: Hierarchical B-spline refinement. ble and non-intuitive behaviour. An alternative would be to ACM Siggraph Comput. Graph. 22(4), 205–212 (1988) linearly interpolate within the hue/saturation disc in HSV, 13. Hjelmervik, J.M., Fuchs, F.G.: Interactive pixel-accurate rendering though also in this case the results are often undesirable. We of LR-splines and T-splines. In: Eurographics (Short Papers), pp. conclude that cylindrical colour spaces are not suitable in the 65–68 (2015) 14. Inkscape Wiki. Mesh gradients (2018). http://wiki.inkscape.org/ context of gradient meshes. wiki/index.php/Mesh_Gradients. Accessed 01 Feb 2018 Results of using the other colour spaces are shown in 15. Lai, Y.-K., Hu, S.-M., Martin, R.R.: Automatic and topology- Fig. 18. Note that this overview does not include linear sRGB, preserving gradient mesh generation for image vectorization. In: as it is a linear transformation of CIEXYZ and therefore ACM Trans. Graph., vol. 28, p. 85. ACM (2009) 16. Li, X.-Y., Ju, T., Hu, S.-M.: Cubic mean value coordinates. ACM produces the same interpolation results. CIELUV, CIELAB Trans. Graph. 32(4), 126:1–126:10 (2013) and sRGB produce slightly darker results than CIEXYZ as 17. Liao, Z., Hoppe, H., Forsyth, D., Yu, Y.: A subdivision-based rep- they include a nonlinear gamma correction. Dark regions resentation for vector image editing. IEEE Trans. Vis. Comput. Graph. 18(11), 1858–1867 (2012) can occur when interpolating in sRGB between markedly 18. Lieng, H., Kosinka, J., Shen, J., Dodgson, N.A.: A colour interpo- different hues. Both CIELUV and CIELAB seem to be suit- lation scheme for topologically unrestricted gradient meshes. In: able spaces for colour interpolation, which makes sense as Computer Graphics Forum, vol. 36, pp. 112–121. Wiley (2017) they were specifically designed to be perceptually uniform. 19. Lieng, H., Tasse, F., Kosinka, J., Dodgson, N.A.: Shading curves: Of these two, CIELAB introduces some undesirable pur- vector-based drawing with explicit gradient control. In: Computer Graphics Forum, vol. 34, pp. 228–239. Wiley (2015) ple when interpolating from blue to yellow, which makes 20. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., CIELUV the best choice among the colour spaces consid- Salesin, D.: Diffusion curves: a vector representation for smooth- ered. shaded images. ACM Trans. Graph. 27(3), 92:1–92:8 (2008) In practice, the difference between rendering in the default 21. PDF Association. ISO 32000-2 (PDF 2.0). Technical report, Inter- national Organization for Standardization, (2017) sRGB colour space and CIELUV is hardly perceptible in our 22. Price, B., Barrett, W.: Object-based vectorization for interactive experience. The reason is that the difference in colour of image editing. Vis. Comput. 22(9), 661–670 (2006) adjacent control points is usually much less extreme than 23. Reinhard, E., Khan, E.A., Akyuz, A.O., Johnson, G.: Color imag- depicted in Fig. 18, and moreover, the patches are typically ing: fundamentals and applications. CRC Press, Boca Raton (2008) 24. Richardt, C., Lopez-Moreno, J., Bousseau, A., Agrawala, M., Dret- much smaller. In addition, conversion between sRGB and takis, G.: Vectorising bitmaps into semi-transparent gradient layers. CIELUV (on the GPU) is non-trivial. For these reasons, our In: Computer Graphics Forum, vol. 33, pp. 11–19. Wiley (2014) current implementation uses sRGB. 123 960 P. J. Barendrecht et al. Jonathan Hogervorst received his 25. Sun, J., Liang, L., Wen, F., Shum, H.-Y.: Image vectorization using B.Sc degrees in Artificial Intel- optimized gradient meshes. In: ACM Trans. Graph., vol. 26, p. 11. ligence and Computing Science ACM (2007) from the University of Gronin- 26. Sun, X., Xie, G., Dong, Y., Lin, S., Xu, W., Wang, W., Tong, X., gen, the Netherlands in 2017. He Guo, B.: Diffusion curve textures for resolution independent texture plans on following the M.Sc in mapping. ACM Trans. Graph. 31(4), 74 (2012) Security & Network Engineering 27. W3C SVG Working Group. Scalable Vector Graphics (SVG) 2 at the University of Amsterdam, (2018). https://www.w3.org/TR/SVG2/. Accessed 01 Feb 2018 the Netherlands. 28. Xia, T., Liao, B., Yu, Y.: Patch-based image vectorization with automatic curvilinear feature alignment. In: ACM Trans. Graph., vol. 28, p. 115. ACM (2009) 29. Xiao, Y., Wan, L., Leung, C.-S., Lai, Y.-K., Wong, T.-T.: Example- based color transfer for gradient meshes. IEEE Trans. Multimed. 15(3), 549–560 (2013) 30. Xiao, Y., Wan, L., Leung, C.S., Lai, Y.-K., Wong, T.-T.: Optimization-based gradient mesh colour transfer. In: Computer Graphics Forum, vol. 34, pp. 123–134. Wiley (2015) Jirí ˇ Kosinka is an Assistant Profes- 31. Zhou, H., Zheng, J., Wei, L.: Representing images using curvilinear sor in the Scientific Visualization feature driven subdivision surfaces. IEEE Trans. Image Process. and Computer Graphics research 23(8), 3268–3280 (2014) group at the Johann Bernoulli Institute of the University of Groningen, the Netherlands. He received his Ph.D. degree from Pieter J. Barendrecht obtained Charles University in Prague in his B.Sc and MSc degrees in 2006. His research interests Mechanical Engineering from include computer graphics, geo- Eindhoven University of Technol- metric modelling, and computer- ogy, the Netherlands. He is cur- aided design. rently pursuing a Ph.D. in Com- puter Graphics at the University of Groningen, the Netherlands, focusing on the application of splines in vector graphics, numer- ical analysis and 3D modelling. Martijn Luinstra is a Comput- ing Science student at the Uni- versity of Groningen, the Nether- lands. He plans on attending the M.Sc in Computing Science at the University of Groningen, with focus on computer graphics and data science.

Journal

The Visual ComputerSpringer Journals

Published: May 23, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off