2 3 Printer-model-based halftoning Advanced Topics in Digital

2. 3 Printer-model-based halftoning Advanced Topics in Digital Halftoning – 17 -19 October 2016

Outline • Halftoning with embedded printer models – Electrophotographic – Inkjet • Black-box model Advanced Topics in Digital Halftoning – 17 -19 October 2016

Digital marking engine technologies: laser electrophotographic • text Architecture of laser electrophotographic printer Typical low-end laser electrophotographic printer: HP Laser. Jet M 252 dw $249. 99 List Instability of electrophotographic process Advanced Topics in Digital Halftoning – 17 -19 October 2016

Commercial/industrial scale electrophotographic printing HP Indigo Press 3050 2, 000 4 -color sheets/hr. HP Indigo Press 30000 4600 3 -color sheets/hr. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Electrophotographic (EP) print models for halftoning [Baqai and Allebach, 2001; Goyal and Allebach, 2008] Outline • • EP print modeling approaches Print characterization Equivalent grayscale image Using equivalent grayscale image in DBS Advanced Topics in Digital Halftoning – 17 -19 October 2016

EP print characterization approaches • Hard circular dot overlap model – Early use by Roetling and Holladay (1979), Stucki (1981), and Stevenson and Arce (1985) – Popularized and extended to tabular approach and color by Pappas et al (1991, 1993, 1997) – Simplified center-offset approach developed by Wang et al (1994) – Widely used in color halftoning • Tone compensation – Comparative study (Rosenberg, 1993) Advanced Topics in Digital Halftoning – 17 -19 October 2016

EP print characterization approaches (cont. ) • Stochastic models – Micro-grid-based random toner particle models (Lin and Wiseman, 1993; Flohr and Allebach, 1993) – Density-based random toner particle models (Flohr, Allebach, and Bouman, 1994) • Scattering models – Bulk properties-based (Yule and Nielsen) – Point-spread-function-based (Kruse and Gustavson, 1996) Advanced Topics in Digital Halftoning – 17 -19 October 2016

Equivalent grayscale image • Image rendered on print • Equivalent grayscale image summarizes effect of dot overlap within each device-addressable cell Bit Map Detailed model for rendered halftone Advanced Topics in Digital Halftoning – 17 -19 October 2016

Representing the 3 x 3 Dot Configurations Weights assigned to binary-valued pixels in computing decimal representation (minus one) 0 0 0 1 0 1 0 0 0 0 0 0 21 22 23 24 25 26 27 28 1 0 0 20 1 2 3 Advanced Topics in Digital Halftoning – 17 -19 October 2016 4 1 ………. 1 1 1 1 512

Print characterization Step 1. Print patterns with fudicial marks Step 2. Segment out fudicials and compute centroids. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Print characterization (cont. ) Step 3. Partition cells, segment dot patterns (yellow), stack dot patterns, and compute statistics Pixel size Advanced Topics in Digital Halftoning – 17 -19 October 2016

Sample dot statistics Minimum Variance Pattern Avg. = 0. 76, Std. Dev. = 0. 089 Advanced Topics in Digital Halftoning – 17 -19 October 2016 Maximum Variance Pattern Avg. = 0. 58, Std. Dev. = 0. 480

Generating equivalent gray scale image Halftoning f[m, n] Binary halftone pattern LUT from Scan Analysis g[m, n] (Equivalent grayscale image) Advanced Topics in Digital Halftoning – 17 -19 October 2016 Indices of dot configuration

Use of equivalent gray-scale image in DBS • Replace binary digital halftone grayscale image • Perceived error with printer model by equivalent • Efficient evaluation of effect of trial changes is still possible, but with greater computational complexity Advanced Topics in Digital Halftoning – 17 -19 October 2016

Error metric • Assume that for a fixed bit map independent of for , . is statistically • First term is metric for deterministic DBS, based on mean error. • Second term penalizes use of unstable binary patterns. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Simulated printer output Artifact probably due to tone-correction DBS with no printer model DBS with hard circular dot model Advanced Topics in Digital Halftoning – 17 -19 October 2016

Simulated printer output (cont. ) DBS with no printer model Advanced Topics in Digital Halftoning – 17 -19 October 2016 DBS with hard circular dot model

Digital marking engine technologies: inkjet • text Typical low-end inkjet printer: HP Desk. Jet 1122 Printer $29. 95 List Architecture of low-end moving carriage inkjet printer Page-wide array used with HP Office. Jet Pro products Advanced Topics in Digital Halftoning – 17 -19 October 2016 Characteristics of inkjet printed dots Illustration of print head architecture and print masking concept

Commercial/industrial scale inkjet printing HP T 400 color inkjet web press – 800 ft. /min. HP Scitex flatbed press (UV cured ink) Advanced Topics in Digital Halftoning – 17 -19 October 2016

Inkjet modeling issues 3 Most printer model-based halftoning work has focused on EP printers 3 IJ printers do render dots having a nearly hard, ideal profile, and much more stable than those rendered by EP printers * However, there exist artifacts which are unique to or more significant in inkjet printing process » ink coalescence (firing adjacent nozzles simultaneously) » satellites (firing the nozzle at higher frequency than they can handle) » random dot placement errors (geometrical error in print head) Advanced Topics in Digital Halftoning – 17 -19 October 2016

Overview of inkjet printing process Multiple-pass printing and print mask – The pen visits each pixel location more than once » multiple-pass printing is usually used to prevent artifacts such as the ink coalescence and the satellites – print mask provides extra logic control; whether the pen should put a drop in this pass or not Vertical position of pen for the 1 st pass 1 0 1 0 1 0 1 0 1 0 Pen Sweep Direction 1 0 1 0 1 0 1 0 1 0 Vertical position of pen for the 2 nd pass Advanced Topics in Digital Halftoning – 17 -19 October 2016 Media Advance Direction

Overview of inkjet printing process (cont. ) Nozzle geometry cyan magenta yellow Front view Intended target Nozzle columns Drop trajectory Nozzle columns Nozzle plate Top view Silicon Ink feed slot Advanced Topics in Digital Halftoning – 17 -19 October 2016 Silicon

Modeling specifics Objectives – Development of an inkjet printer model for the target printer – Incorporation of the inkjet printer model into halftoning algorithms Target printer – HP Ink. Jet 970 Cx printer Target printing mode – resolution: 600 dpi – printing direction: uni-directional printing mode – printing speed: 10 inch/sec – 6 -pass printing mode – each of even and odd raster lines is printed with different nozzle column Advanced Topics in Digital Halftoning – 17 -19 October 2016

Print Characterization Test pattern (600 x 600) Printout (scanned) + Segmentation map) even odd * scanned@4000 dpi Advanced Topics in Digital Halftoning – 17 -19 October 2016

Print Characterization (cont. ) Dot profile Estimate the dot profile based on the measured the absorptance of each segmented cell Dot displacement ref. line * ref. line = averaged centroid * displacement = centroid - ref. line Advanced Topics in Digital Halftoning – 17 -19 October 2016

Printer Characterization (cont. ) mean dot profile Advanced Topics in Digital Halftoning – 17 -19 October 2016 std. dev. dot profile

Print Characterization (cont. ) horizontal dot displacements for even raster Advanced Topics in Digital Halftoning – 17 -19 October 2016 horizontal dot displacements for odd raster

Print Characterization (cont. ) vertical dot displacements for even raster Advanced Topics in Digital Halftoning – 17 -19 October 2016 vertical dot displacements for odd raster

Printer model-based DBS (Conceptual) bitmap real printer EQGS sampler HVS Error original image ideal printer HVS – DBS seeks to minimize the statistical average over the ensemble of all possible realization of e Advanced Topics in Digital Halftoning – 17 -19 October 2016

Error measure The first term on RHS: deterministic measure based on the mean value of The second term on RHS: penalizes the random fluctuations between dots We used two error measures: 1 st term only and 1 st and 2 nd terms Advanced Topics in Digital Halftoning – 17 -19 October 2016

Experiments How well DBS can exploit the printer model? * Experimented with simulated printers How well the model characterizes the printer? * Experimented with the actual printer Advanced Topics in Digital Halftoning – 17 -19 October 2016

Simulated printer for testing how DBS can exploit printer model digital halftone bitmap Printer model low resolution rendered bitmap Up sampler (high resolution) rendered bitmap Simulated printer digital halftone bitmap low resolution rendered bitmap Advanced Topics in Digital Halftoning – 17 -19 October 2016 high resolution rendered bitmap

Simulated printer: never-centered printer – Dot profile: ideal square dot – Dot displacement: fixed dot placement » even raster: shifted by X/2 » odd raster: shifted by -X/2 Advanced Topics in Digital Halftoning – 17 -19 October 2016

Simulated printer: never-centered printer* DBS with no printer model *Even raster shifted by X/2; odd raster shifted by -X/2 Advanced Topics in Digital Halftoning – 17 -19 October 2016 DBS with printer model

Real printer: sample HP IJ 970 Cx outputs DBS with no printer model DBS with printer model (no covariance term) Advanced Topics in Digital Halftoning – 17 -19 October 2016 DBS with printer model

Predicting printed absorptance from the digital halftone: inputs to the black-box model Advanced Topics in Digital Halftoning – 17 -19 October 2016

Structure of the black-box model Advanced Topics in Digital Halftoning – 17 -19 October 2016

Why We Need to Consider a Larger Neighborhood Than 5 x 5? • This local influence can be attributed to following facts: – the spot size of the laser write beam is larger than a single printer-addressable pixel – the complex field interactions that are set up by the charge distribution on the photoconductor and in the toner in the gap between the photoconductor and the developer, and how this influences development – the further spreading of toner during the transfer and fusing processes – optical scattering of incident light within the media. Advanced Topics in Digital Halftoning – 17 -19 October 2016 38

Why We Need to Consider a Larger Neighborhood Than 5 x 5? Measured central pixel absorptance as a function of neighborhood’s size Dispersed-dot halftone generated using DBS algorithm Portion of test page Conclusion: ØNeighborhood larger than 5 x 5 can strongly influence the central pixel absorptance ØWe choose 45 x 45 as the size of neighborhood We also have verify the influence only depends on the gray level of the neighborhood, not the distribution Advanced Topics in Digital Halftoning – 17 -19 October 2016

How do We Build the Black Box Models? Print the test pages at 812. 8 dpi Scan at 2400 dpi Design test page Epson Expression 10000 XL HP Indigo Press 5000 Advanced Topics in Digital Halftoning – 17 -19 October 2016 40 Scanned image analysis Build printer model

How do we train the model? 5. 1. 2. 3. 4. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Scanned Image Analysis The centroid is calculated based on the spatial distribution of toner absorptance throughout its corresponding mask region The 0. 5 pixel offset shifts the coordinates for each pixel to the center Calibrated scanned image Locate the centroid of each fiducial mark Locate all pixels that have 45 x 45 surrounding Advanced Topics in Digital Halftoning – 17 -19 October 2016 42 Estimate absorptance for all pixels within the region of interest Statistics data for black box models

Scanned Image Analysis Calibrated scanned image Locate the centroid of each fiducial mark Locate all pixels that have 45 x 45 surrounding Advanced Topics in Digital Halftoning – 17 -19 October 2016 43 Estimate absorptance for all pixels within the region of interest Statistics data for black box models

Experimental results – sample images Scanned image Digital ULM 5 x 5 prediction ULM 5 x 5 error image M 45 x 45 c 3 b prediction M 45 x 45 c 2 a error image Gray level 96/255 Advanced Topics in Digital Halftoning – 17 -19 October 2016 M 45 x 45 c 3 b error image

10 9 8 7. 33 7. 68 Model fit 7 RMSE (%) Experimental results – error statistics 6 5. 06 5 4. 77 4 3 2 1. 31 1. 85 1 0 ULM 5 x 5 * *Absorptance units are on a scale of 0 (white) to 1 (black) Advanced Topics in Digital Halftoning – 17 -19 October 2016 M 45 x 45 Class 2 a M 45 x 45 Class 3 b