RSD-TR5-83 M UL TI-B US COMPA TIBLE FRAME GRABBER J. L. Turney J. V. Hill May 1983 Center for Robotics and Integrated Manufacturing Robot System Division COLLEGE OF ENGINEERING THE UNIVERSITY OF MICHIGAN ANN ARBOR, MICHIGAN 48109

1 1. INTRODUCTION A vision system generally consists of a camera, a digitizer, a computer, and a video interface such as a frame grabber. In the system the camera and digitizer provides digital video data at a rate of several megabytes per second. For example, the General Electric TN2500 digitizing camera provides an 8-bit pixel (picture element) at a rate of 4.5 million pixels per second. At this rate of incoming data a computer cannot directly perform meaningful processing on the data. Therefore, vision systems, contain a hardware interface between the camera and the computer to capture incoming video data and to hold the data until the computer has time to process the captured data. This is the role of a frame grabber. Essentially, a frame grabber is a dual port memory with one port of the memory connected to the camera and the other port connected to the computer. A video frame of data from the camera is digitized and brought in, i.e. "grabbed", through one port and held in the frame grabber's memory until the computer has time to process the data, which it accesses through the second port. Once the image data has been captured it may be thresholded, or edges in the image data may be determined, or any other processing may be applied to it. In the Robotics Lab, we employ several Intel 8086 based boards as dedicated processors for vision applications. We decided to build a Multi-Bus1 compatible frame grabber board which would operate in conjunction with our General Electric TN2500 digitizing camera, in order to provide ourselves with a system component which would occupy little space, would derive its power and timing from existing systems, and which above all would be inexpensive to reproduce. This report discusses the result of out efforts -- a simple and relatively inexpensive frame grambber (cost $600). Multi-Bus is a registered trademark of Intel Corporation.

2 2. OVERVIEW Fig. 1 presents an overview of the board. Every 220 ns, a new pixel of digitized video information is brought in from the TN2500 digitier (shown in the upper right) and latched into one of the buffers, B53, B52, B51, or B30 in order. The frame grabber's memory requires times greater than 220 ns to write data, so thedata is temporarily buffered in B53, B52, B51, and B30. One cycle of this buffering process consists of filling the four buffers. Each cycle takes 880 ns, 220 ns for each new pixel to be brought in and written to a buffer. When the last buffer B30 is written, the other three bytes stored in B53-51 are then output simultaneously to buffers B33-31 after which the contents of B33-30 are output to the frame grabber's memory when the memory is ready to accept data. As mentioned above 880 ns of time passes between the times that the buffers B33-30 are filled. During the first half this time, 440 ns, the frame grabber's memory is written from the buffers B33-30. The 32 bits in the four buffers B33-30 are written simultaneously. During the later half of the 880 ns buffering time B33-30 are not connected to the frame grabber's memory, and the computer is allowed access to memory. The 880 ns provided by buffering is thus time sliced between the camera and the computer. The memory is interfaced to the computer through bidirectional gates B62-60. The computer writes into memory through B13-10 and outputs from memory through B23-20. Latches B43-B40 provide a interface to a video monitor, whereby the grabbed frame may be displayed. The sync signals to run the monitor are taken from the TN2500 while the actual video information does not come directly from the camera but rather is routed from the frame grabber's memory. A frame can be grabbed and displayed on the monitor without the need for onboard video timing generation logic

3 6\1)x8 I CKX I wGx 8 I. ERA 8532 36 I I t1 I -- -H _ _ __ ____|_ I____ _____ 803 rpkxag 6'/xx I 6YKX8 (VKxs AicrTPJlnoN ME4,9BY7-E I -RY) CO 7OL VEI S l V ITIDEO I- --— $ -L 1- I LOG- I I-CO TTRoLO 062' L B613 | 0 B-N-I-60; _. ^MULT/IUS /DDRSSING _ lMuL/i7eus Dm4 MOJ'LT/6IuS CONIrcL__ Figure 1. Overview of frame grabber. since the frame grabber steals the necessary sync signals from signals provided by the TN2500 digitizer. In the present configuration of the board, data coming out of the frame grabber to the monitor is thresholded before it is output to the monitor. In the future an D/A

4 converter will convert the digital data back to analog for a gray level display. The threshold is software selectable. Automatic addressing for the memory is generated using signals from the TN2500 digitizer (see left side) during the time the camera has access to the memory. The computer can also address the memory during its 440 ns time slice. The frame grabber actually has 4 pages consisting of 256 x 256 bytes each of memory. Any one of these four is software selectable. Since the computer can write into each page. One page could be used to display data while the computer is operating on a different page. Table 1 provides a price breakdown on the cost of the board, and Fig. 2 shows the actual layout of the board. 3. PROGRAMMING The frame grabber, as it is presently configured, resides in Multi-Bus addresses 40000H to 7FFFFH. Jumper J1 on page A1 of the appendix can be altered to relocate the frame grabber. memory chips $ 320 logic chips 70 sockets 70 board 130 Total $ 590 Table 1. Price breakdown for frame grabber.

5'~ —-—'n EL S3 _t~___-____ VI Li L-i I A B!o Figure 2. Board layout. ThTCL Si3C -'10-5 0 The frame grabber is control by an 8255 PlO. In order to initialize the PlO, the byte 99H must be written to address location C206H. This location may be relocated by alterating J2 and J4 displayed on page A1. A frame is grabbed by writing OOH to address location C300H. The lowest bit of input from location C200H will be 1 during the frame grab and will go to zero once the frame has been captured. This bit may be polled to determine if the frame grab has been completed. The monitor image is thresholded by a byte threshold set at location C202H after the PlO has been initialized. This threshold only affects the monitor image but

6 does not affect the memory of the frame grabber or the data read from the memory to the computer. The computer must threshold the data itself once it has captured it. Appendix B gives a program for grabbing a frame of data and converting it to a runlength encoding. 4. DETAILED OPERATION In the schematics shown in Appendix A inputs are listed on the left side of the page while outputs are listed on the right. Numbers next to the inputs correspond to the page from which the input is obtained. For example the n and"3" next to MXACK/ indicates that the input MXACK/ is obtained from a similarly labeled output MXACK/ which is shown on the left side of page 3. The numbers next to an output correspond to the pages on which the output appears as an input. The special prefix of "MB" indicates that the connection is to the Multi-Bus line rather than to a connection on another page. Page Al displays the address decode logic which decodes address from the computer to the board off the Multi-Bus. L21 selects the memory. Jumper J1 determines which quarter megabyte of the Multi-Bus memory space the memory of the board will occupy. L1 9 and L20 select the PIO. J2 and J3 configure the PlO's location. L19 also provides a frame grab request FGREQ/. L25 provides the required response to an 10 or memory read or write, since each memory access over the MultBus must be acknowledged with a XACK/. Page A2 shows the Multi-Bus data interface. B60-62 are bidirectional transceivers. The 4164 memory chips are dynamic rams with multiplexed address inputs. The desired 16-bit address must be placed on the 4164 memory address lines 8 bits at a

7 time, the lower 8-bit during an RAS/ negative transition and the upper 8-bits during a CAS/ negative transition. L03 provide the RAS/ and CAS/ strobes as well as other signals. MUX is used to multiplex addresses from latches 8 bits at a time. LSTROBE strobes data out of memory into latches for output. L31 MXACK/ output provides the desired Multi-Bus response for a memory access at the correct time. Since the TN2500 digitizer does not provide any addressing information, addresses during the camera's access to memory are generated from the element rate clock, ERC/, using a series of down counters L1 1-1 4 as shown on page A4. Outputs VAO-7 address one of the 256 pixels in each horizontal line of the image. Outputs VA8-F address one of the horizontal lines in the display. During a frame grab VA8 is a different output than during normal monitor display. This is because the TN2500 displays lines in a peculiar way. It displays odd lines twice each during the odd interlace field and even lines twice each during even interlace field. This results in an undesirable overlap of lines. Lines appear in the order 1,2,1,2,3,4,3,4,5,6,5.. Using 11 5 together with L08 and L09 eliminates this problem and allow lines to be appear in the order as 1,,2,2,3,3,4,4,5,5 ~. Page A5 shows the PIO and its connections to both the frame grabbing latches L32 and the thresholding logic L1 7 and L18. The thresholded data is level shifted to provide a simple video output. L1 5 is used to clean up the display. It pulls the output low prior to a horizontal sync to provide a uniform delay to the low caused by the sync, and it blanks the screen when portions of memory which are not filled by the frame grabber are scanned. Page A6 shows how the memory and video address are multiplexed into the memory. Page A7 displays the buffers responsible for latching and holding the video data during the frame grab, while page A8 shows the data interface from memory to the monitor.

8 Pages A10 and A 11 display the data interface from and to the computer bus. Page Al1 shows how the memory chips are connected. In the interest of conserving space, only one example out of a group of eight memory chips is shown representing typical connections for members of each group. Page A1 2 shows how the connector brings signal from the TN2500 digitizer to the board. 5. FURTHER WORK As mentioned before a D/A convertor will be installed in the frame grabber, so the user will have the option to display either the actual frame grabbed or a thresholded version of the frame grabbed. We wish to thank Bob Alsum for doing the bulk of the original design and for doing the tedious job of wire wrapping the board.

9 6. APPENDICES

Vcc Mo 34 ~ A 00 memor — rj 7iwk A MB 32 API v _____1 A_ __ /DDRESS DECODE a XqCK 3LlO _ r —---— o MRE9 3,b to 5 2c —-- O, t3 O MI q3 qDB E _ EN538 Gat.,3.s Mc l 0~ } MHI iq BWC o01, IL-Lo0 5 tf lis. 45 PA, -------- C 0 _. o. c 11 0i AI ~ N -IC STl — 0 -- I QL Z O, S — q C- | A 19~/ | -0I-~ V 1 < 2 13 11 12J 5 P5 o,' 71,6__o6_:r MO 31 CC-rK Hct f IOSEL ^pes —---------— 3 -- - i, 2 QO-'5- Mr M15 I —---— 3 - L _ -,'z Loo,~ ) ~1 MI3 15 IN IT —— s - -> NO.-~.

Cc D1T/ FS3 I TER FACE 8 2 MO1B 73 2 1 lpD 2Z,5,9 MO' 72 -- 2. 5 —--------- M Y 72 1 i - --- ) z, 5,4 MP 7I --- - t —-— 17 —--— 13-_ —-> p~ ZS9 M~ 4 67 _ _-__ Ss _ ->P>(,5^ y'1<7'O l l l - _ —-X - I CD_ _____________ 226,5_ — -O 3 ________,5, M 51 3 5>A Z M~ o Ba- -, I)F MO 2-pr M4 0z1T iq_________________ o'. — 3,0 Ms ^ —----— ^ Wri -— 4III1 —-----------— ~ D4 4 —---- Ho -----— " 62' -s -- o NO 59 V-e4f__ 13 -- ----------------- ---- II 1 0 --— >3HFNSL 2 140 27 BHEN —-------- k Z 0 -'-V2 —----------

|5L ME5MORY STROE5 4- 1hMIG G 3 R 5. 2. -J-' 1 Pp.(. -W u 2 ~ MBE 3,1 LzL # V4S8- - - 7 - -6 c, ~ Q j 5, so I _ Ixl I p^ lbHIo L31 G Bl7- --- _2 P 9,I 14 ___ 12____________ MRT. it *. MT'r CXA. 14 6p CR B8 __ I I l I' I ^ ~~~'12-4 B c F - 5 114^ 114 11 3 - 1,0 2- INH —I I3.Tjj3 M E 6 -- I 5ITX_... IB0 I r r | — --,i ~ 3tJ MUX 6 H I l-rf pt ci pi,r pr or.t —- iV ~.... i —__________, Icsf -- l -_, rO ~5X it 1.x[ -- ---- 1t 75 -- os IAlI — mox _ M/ J_ j i~ lt^ G ______________z. _______ Z________ > __ux

12. EFlZ. EFK ( y5_,5 W,.C. VlE ADoRE S5 GENAERflON AN4 1_____-' 4E; CI B 3 I2, G 5 -— = ^ - l|, 1t — 4 - 7-_-=> vvE 5, I' qp -h I 7 5P FG2q -- -',I -v-uc 6 r^ e p134' - - --- L^ ^ —-------- F1 —-1 C LJI3 qp,_Lg ^'^Tvi'(f.,cjg_ 3 B (?-3'7 - -^rcrii —--— )'V/l 6, toLc Li 1 19C 7 V40 V4 12 ERC-76 It-5 Vec, LII Vee 13 16

RAR q/ME (_R 3B::9' rtiL. inO- ^ uL.-'IvIy H vsc- - A —-- A5 NC T 9v<':c Is k l 2,10 D^-\ \5\ B 3/B v.70n 2,10 Z 371 II 7 zllo 97 2 A P B3 - |3 18- || J, A2 14 A 7 |2 113 53 zio P23PB L 11 10 Di_ __ _ _ __ _ _ z, — I - A Z -JS^-^ > ^ - MB 51 I()oC 5 | RP pc N?V/D I A ( U_2,10_ _ _ P3 )3 F7 __ q —, A_ I a 2,ro D1 D 3 7 5 9 2 Z710 Al, oq M) J)133 —0 > CLW- PR Ii zPI ttL — _ L1I _ _ 7,i3 LI) ORp BO 1t~2,,1 - +~o FI < --—._ —-> i f INI0 LT 3 K1 /D'zr,os9 "J5 E 3 PC; 3 Lz EFBLP ovcC 1 13 13-0' 35 CLR I0Y I- c5 3 MiOE MOZ! 1l9,.1 Rif

13 r'/ VA.-I-,' a — 87 1 -- ^ /96 Lf VAS7 Vqb'1 V43 1 q V4 ---— ____7________3-_ 1 -4- ___ Lt vf ~r W -------— ___ 7r^3 0 ^ tw —-----— ^ v2 ^MfEmoRY PiDRE5s 7INTERFACC 13 l 5 V4A 1 I 1 -- V4 F 174~- FS II/ A It v —--- Bpz ^.\ B~~~~~~~~~~~~~~~~~z~~~~~~B3 q/ v- ro A 7~~~~~~~~'' AI V —- 7I7 ___^ —_____________3 7>3 Mv q 013' q VWAA~,dB53 A^q__ ___________ -S MUK rHr~t1 2t-ADRb —------ 9 ^^$0 ~ ~ ~ ~ -r Q________^ r ^ __ MIS ^ - -- - --- -^ _______- - ^- __3 1^ ___ t'13 y6 A~~ MB^ AM&c- -^ - - - B^< j^_ 7YI9 M 8q(0 AV~ Ko 7 l qi$373 MB'6RE I IC Me^ 5("por -- - - - — __ _i 4 ^, ^ _?41 5f 1A4 3 45 / ^________ _ AP 12 ___ MB q( AV --- _ _____ _ la__ ^7- --- MD9 /10f - --— _________ ^ W C. 7 7 —3 He 11 An -- - ________ ____ - MB VI A —---— _ ___ _ ___ A —--'MRE ---- - _________97 ^ -- --

VIODED OTR IN 7 _V____ __: DV/I I TMDI 11I It VI S IM'DIo 1 12 V632 L i$ MP1E I 3 G R3 j rIOIC 1I ~3IZ vc - — 3?- -- -.53Z 3 7q\s-373 3 - I-MtD D i z. ~ ~ _ ffi, Dz A iZ. ve5 ~ Dip,33 - -B3> Vt^dt —,._ 52 ) ~_____t.,L]~~.. iS<~ |i~.j~ lf' IMPl7 I 70;_ ____J__ z. >P V 1 l XSJ ~..__.7_~3'1-S~ y $ ~ 7 7^S37 ^ ----------------------— ^lrIMt~6 it.- ---— _ 5 _, W,. I MvlI Ii S___-L JS _____ _____ &- < —--------'>IMPF t1 1357.. hPSB,_ —---- -------------— I ir" 1 --—. ]~~~~~~~~~~M(I

3 _ g ll' /1 t w ^,, 7 t_, II O 41HP-I 4 5 -> DIqP (I OOPE4 |- (I o0p0 ---- 1- - - o -1p t lb I OHL)5 1 I_ 1( O^0 13 B -2_ —[_ _, ii, 0t-0 I, —- I I I II OMP - - ii 0)4911 _ II OHD 2, olp I I —a 737,I or,, ~3 — T UT q OMS -- Kl OHPF --- I, o —I B)3 C7 — II OMIT. 3,-s'.... —---— II

Z pD r _ _ __i7_|_ _ _ __|_ IMIDID:I z D- - 1 Y * *" P 13I I II D5 Iq! 79137.3 7- 3Ipi! II 2 PC -- 9 1- 5 li 2 Di z p ----- -- |l3 19l 1 _ __ ____ I _pji_ — iPI I3 i - _ _______1 iMPF)3 II Z2 Di~S=I | ~~~~~~~17 Itl~~~~~~~~~~~~~" MP1 " " ___I'- - I,21 --- ~,,I, 7qLS3/7 6 30 f'113El~ 1 17 16 MP7 II 13' 1,, ------ ______ 7qLS373.:1' "o83 >v85t 3 M0E -- -. - AB 20 M3 WTC —-- V _

, _ )l^'r __p__(5 /fu l22 AT~ OUT ffc 2 ft7-iS F i o-lt-' DP, 2.l,t II OM1IB |r I oWD g ->__ D 2P ~If 0__ —~ - 11 OlPl p t II 014 VIE' ____ O ---------------------- t 11 08 —-------— __ 17.5i --.! —-- -> p5 z II OMDI3 7 (5 09tl1 — - _ _.. 4 5 4 > t} L _I OP10, ie 7 ^ l' - 3 It 3l OMOF --- -Is L 13 Il OMOE _..Il 7:_ I I1 OM-C)- E - B22 1, OMOC I1 010 -a 7-379 11 0C107 -- _LE I\ OMP7 ------- -n' 1j I iI oM —- _ ---—.-, I, OmC5, I OMDLf.,, i: OtF -i, OMP3- --— _______3^ -- qL5 II OMPJ - 2 Al.,

s3 —------ /4 MOF Y C ONNECTIONS C(, A —---- - - AM (p Mh7i2,rq4 75 t 6 ccl~~~~~~~b- IqC5 me, m b ~~~~~~~~~~'I I~~~~ A9 6 M43 6 —- MOO-M07 1 c MAZ ----- - O cc & HAl -- - AG~ 3j T ot C p RAS -T- --— | 4 7' tntI7 - A-7 — A5i 9 ch-~ —----------------— OMe c OM D10 -OmPh7 3 - N.C c__O MIO-M17 7, 7 onelfr FHPIO- M~ff --- ----- ---- ~. p ^//ot RAs -L3 QOS 7 Al p 9. Ode c7 CA o-lr mc. fj ~~ i~ —---— \ t-L^ 41) <?C —----------------— efO^ O^ t,~ MZa —M7 7j 7 ohedfJtW-l& —-- - _ 2rf Pm AP 3 orA - - 3 P5 3I, ^ — a-47 w i 45 9 r iie c( avoDM7 3 ^ — S~ 13 /K, /_ne IC_____ ___ \30- HPt Al - 1 co-" 0 PO 110l 7, cw J -im^-ino7 --- ---- D Z 5 ~

Z T CONNECTOR /JTT Al/2 J ]Pur FROM 1/V2500 1 It)>V83 7?_o.n VB3 *7 Is >Vos 7 9 > ~VG 7 Z —> V8a 7 ~~~~~~6 > ~~~~~~5,X 3 ----------------- 7 LS-an- 7 IoI I O> 5F 5 1#3 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~I, fYSYNC e |5 LA |1 IB | 21s Z

SERIES-111 PL/M-86 V2.0 COMPILATION OF MODULE ENCODE NO OBJECT MODULE REQUESTED COMPILER INVOKED BY: PLM86.86:F2:ENCODE.PLM NOOBJECT /*** This program performs a run-length encoding on / a frame stored in the frame_grabber's memory. /**0 It is written to be executed on an Intel 86/12A / single board computer, using the Intel iSBC 957B /000 monitor and an Intel Series III as the console. */ 1 encode: do; 2 1 declare (transit,white,threshold) byte; 3 1 declare (i,j,indx,indx2) word; 4 1 declare true 1 i terally'1'; 5 1 declare false literally'0'; /ee encode contains the encoded image **/ 6 1 declare encode(1024) byte; /** frame_grabber memory starts at location 40000H 0*/ 7 1 declare buffer(Offffh) byte at (40000h); /*** external procedures provided by the 957B monitor 01e/ 8 1 ci: procedure byte external; 9 2 end ci; 10 1 co: procedure(char) external; 11 2 declare char byte; 12 2 end co; 13 1 exit: procedure external; 14 2 end exit; /s*0 ascii8co is used to print a byte.,;;e on the console. *ee/ /*** (value is displayed in hex) **"/ 15 1 asciiSco: procedure(char) public; 16 2 declare (temp,char) byte; 17 2 temp=shr(char,4); 18 2 if temp>=10 th.en temp=40h or (temp-9); 20 2 else'temp=30h or temp; 21 2 call co(temp); 22 2 temp=0fh and char; 23 2 if temp>=10 then temp=40h or (temp-9); 25 2 else temp=30h or temp; 26 2 call co(temp); 27 2 end asci ico;

/eee get threshold level from console / / /** print prompt **/ 28 1 cat I co(3fh); /** input hex value g*/ 29 1 indx=ci; call co(indx); 31 1 if indx>39h then indx=indx+9; 33 1 threshold=shl((Ofh and indx),4); 34 1 indx=ci; call co(indx); 36 1 if indx>39h then indx=indx+9; 38 1 threshold=threshold or (Ofh and indx); /** display value to validate input **/ 39 1 call asciico(threshold); 40 1 call o(0d'h); call co(Oah); /*** instruct the frame-grabber to get a frame 0**/ 42 1 output(0c300h)=0; /*** set-up frame_grabber's monitor output **-/ 43 1 outpu t(0c206h)=99h; 44 1 output(0c202h)=threshold; /**e wait for complete grab **/ /e do while not(input(0c200h)); end; 45 1 / call 1 time(4000); 46 1 i n dx2=0; /** encode rows 5 to 230 and columns 28 to 250 to avoid camera */ /** noise at the frame edges */ 47 1 do i=5 to 240; 48 2 indx=i 256; 49 2 encode(indx2)=i; 50 2 white=true; 51 2 transit=false; 52 2 do j=28 to 250; 53 3 if ((buffer(indx+j)<threshold) and white) or ((buffer( indx+j)>threshold) and not(white)) 54 3 then do; 55 4 indx2=indx2+1; 56 4 encode(indx2)=j; 57 4 white=not(white); 5? 4 transit=true; 59 4 end; 60 3 end; 61 2 if not(white) 62 2 then do; 63 3 indx2=indx2+1; 64 3 encode(indx2)=231; 65 3 end; 66 2 if transit 67 2 then do; 68 3 indx2=indx2+1; 69 3 encode(indx2)=0ffh; 70 3 indx2=indx2+1; 71 3 end; 72 2 end; /e* flag end of encoded frame */ 73 1 encode(indx2)=Offh; /** display encoding on the console */ 74 1 j=0; 75 1 do i=0 to indx2; 76 2 call asciico(encode(i)); call co(20h); 78 2 j=j+l; 79 2 if j=16 then do; 81 3 call co(0dh); 82 3 call co(0ah); 83 3 j=0; 84 3 end 85 2; i 86 1 call exit; 87 1 end;