Never

(a) Determine the classes of this Markov chain and, for each class, determine whether it is recurrent or transient.

States 1 and 3 are accessible from each other (p31 = 1 and p13 = 1), but noother states are accessible from these states (p1j = 0 and p3j = 0 for j = 0, 2, 4). Therefore, states 1 and 3 communicate and form one class of the Markov chain. Upon entering either state, the processwill return to that state in two steps, so {1, 3} is a recurrent class.
State 0 is accessible from state 4 (p40 = 0.8), state 2 is accessible from state 0 (p02 = 0.5), and state 4 is accessible fromstate 2 (p24 = 0.7), so each of these states is accessible from each of these other states. Therefore, states 0, 2, and 4 communicate and form a second class of the Markov chain. The process can movefrom any of these states to state 1 or state 3, in which case the process never would return to states 0, 2, and 4 again. Therefore, {0, 2, 4} is a transient class.

(b) For each of the classesidentified in part (a), determine the period of the states in that class.

We calculate P(2) and P(3).

P(2) = P*P = .

P(3) = P(2)*P = .

Since p11 = p33 =0 and p(2)11 = p(2)33 = 1, the class {1, 3} has period 2.
Now note that p00 = p22 = p44 = 0, p(2)00 = p(2)22 = p(2)44 = 0, and p(3)00  0, p(3)22  0, p(3)44  0. This indicates that the class{0, 2, 4} has period 3.

Example for Section 16.5
A soap company specializes in a luxury type of bath soap. The sales of this soap fluctuate between two levels — "Low" and "High"— depending upontwo factors: (1) whether they advertise, and (2) the advertising and marketing of new products being done by competitors. The second factor is out of the company's control, but it is trying todetermine what its own advertising policy should be. For example, the marketing manager's proposal is to advertise when sales are low but not to advertise when sales are high. Advertising in any quarter of...