% poker.m
% CS 8803 PGM  Intro to Probabilistic Graphical Models
% Spring 2005
% Jim Rehg
%
% Problem 7
% Bayesian inference in a simplified poker game, taken from
% Sections 2.1.5 and 2.2.3 in "Bayesian Networks and Decision
% Graphs" by Finn V. Jensen, Springer 2001. The relevant portions
% of this text will be handed out in class.
%
% INSTRUCTIONS:
% Part (a): Fill in the missing code below (where it says "HW: FILL IN...")
% Part (b): See end of file
%

% Specify DAG structure
%
%      H0
%      | \
%      |  FC
%      | /
%      H1
%      | \
%      |  SC
%      | /
%      H2
%
N = 5;
dag = zeros(N,N);
H0 = 1;
FC = 2;
H1 = 3;
SC = 4;
H2 = 5;
dag(H0, [H1, FC]) = 1;

% HW: FILL IN THE REST OF THE COMMANDS TO SPECIFY THE DAG STRUCTURE ==============

% Specify the different types of hands
% H0 and H1 use the 9 categories below. H2 removes C2, S2 and adds A2 (2 aces)
NO = 1; % Nothing special
A1 = 2; % One ace
C2 = 3; % Two consecutive numbers (1,2 or 5,6 etc.) Not a scoring hand, but important during drawing.
S2 = 4; % Two of same suit. Not a scoring hand, but important during drawing.
V2 = 5; % Pair (of same value)
FL = 6; % Flush (3 of same suit)
ST = 7; % Straight (3 of consecutive numbers)
V3 = 8; % 3 of kind
SFL = 9; % Straight flush

% Specify actions
D0 = 1; % Discard 0
D1 = 2; % Discard 1
D2 = 3; % Discard 2
D3 = 4; % Discard 3 - not available on SC

% Create BNet shell
number_states = [9 4 9 3 8];
bnet = mk_bnet(dag, number_states);

% Fill in CPT's from Jensen p. 51
% Initial distribution over possible hands (prior)
bnet.CPD{H0} = tabular_CPD(bnet, H0, [0.1672 0.0445 0.0635 0.4659 0.1694 0.0494 0.0353 0.0024 0.0024]);

% P(H1|FC,H0) - probability of second hand given first card change. 9 x 4 = 36
% combinations. However many are ruled out by strategy and can be assigned arbitrarily.
tbl = zeros(9,4,9); % space for completed table: H0,FC,H1

% Assign to each combination the default distribution in which NO has probability 1. While this is
% not accurate (e.g. the combination V3,D1 will result in a V2 hand at least) the assumption is
% that these combinations are not going to occur in practice, so accuracy is not important.
tbl(:,:,NO) = 1.0;

% Now override the default for the 9 important combinations from Table 2.7 in Jensen
tbl(NO,D3,:) = [0.1583 0.0534 0.0635 0.4659 0.1694 0.0494 0.0353 0.0024 0.0024];
tbl(A1,D2,:) = [0 0.1814 0.0681 0.4796 0.1738 0.0536 0.0383 0.0026 0.0026];
tbl(C2,D1,:) = [0 0 0.3470 0.3674 0.1224 0 0.1632 0 0];
tbl(S2,D1,:) = [0 0 0 0.6224 0.1224 0.2143 0.0307 0 0.0102];
tbl(V2,D1,:) = [0 0 0 0 0.9592 0 0 0.0408 0];
% HW: FILL IN THE REMAINING ENTRIES BELOW ==================
tbl(FL,D0,:) = [];
tbl(ST,D0,:) = [];
tbl(V3,D0,:) = [];
tbl(SFL,D0,:) = [];
bnet.CPD{H1} = tabular_CPD(bnet, H1, tbl);

% P(H2|SC,H1) - similar to the above but for second change, and over the 8 possible values of
% the final hand: NO, A1, V2, A2, FL, ST, V3, SFL
tbl2 = zeros(9,3,8);  % (H1,SC,H2)
% Defalt distribution
tbl2(:,:,NO) = 1.0;
% Distributions from Table 2.8 in Jensen
tbl2(NO,D2,:) = [0.5613 0.1570 0.1757 0.0055 0.0559 0.0392 0.0027 0.0027];
tbl2(A1,D2,:) = [0 0.7183 0.0667 0.1145 0.0559 0.0392 0.0027 0.0027];
tbl2(C2,D1,:) = [0.5903 0.1181 0.1154 0.0096 0 0.1666 0 0];
tbl2(S2,D1,:) = [0.5121 0.1024 0.1154 0.0096 0.2188 0.0313 0 0.0104];
tbl2(V2,D1,:) = [0 0 0.8838 0.0736 0 0 0.0426 0];
% HW: FILL IN THE REMAINING ENTRIES BELOW ====================
tbl2(FL,D0,:) = [];
tbl2(ST,D0,:) = [];
tbl2(V3,D0,:) = [];
tbl2(SFL,D0,:) = [];
bnet.CPD{H2} = tabular_CPD(bnet, H2, tbl2);

% P(FC|H0) - Distribution over possible first discards given initial hand (H0). Implements strategy
% on Jensen p. 50
% HW: FILL IN THIS CPT AND STORE IN bnet ==============

% P(SC|H1) - Distribution over second discards given H1. Same strategy as FC|H0, but with one
% fewer discard possible.
% HW: FILL IN THIS CPT AND STORE IN bnet ===============

% Problem 5, Part (d)
% Answer the following questions
%
% i) Sanity Check. 
% Compute the following distributions: P(H2|H0=FL), P(H2|H1=V2,SC=D1), P(H2|H0=NO,H1=V2,SC=D1),
% P(H1=A1). For each one, explain why the reported answer is correct.
%
% ii) Betting Tool
% Assume that there are three rounds of betting in the game. First, before any cards are dealt, each
% player "antes" by placing a fixed amount in "the pot" (the stakes for which the game is being
% played). After FC there is a first round of betting. After SC is a second round of betting. In each
% round of betting, the player who did not deal the cards makes a bet, and the second player
% must either match the bet or "fold", which means that they forfeit the hand and the contents
% of the pot go to the other player. All bets go into the pot. If both players are still in the
% game at the end of SC, then they compare their hands. The person with the best hand according
% to the order {NO, A1, V2, A2, FL, ST, V3, SFL} wins the pot. (In practice ties would be broken
% by a high card).
%
% Explain how to use the Bayes net model to determine whether to "stand" (match the other players
% bet) or "fold" (forfeit the hand) at each round of betting, based on observations of FC and
% SC. Write the necessary Matlab code.
%
% EXTRA CREDIT (Worth +1%)
%
% Simulate 10 repetitions of a simplified poker game as follows: Each player starts the game with
% $20. Ante is $1 and there is a single round of betting after SC (i.e. no betting after FC) in
% which each player either bets $1 or folds. Player 1 will always bet $1. Player 2 bets if the
% following inequality is true, otherwise he folds: E(H2|FC,SC) < M2, where M2 is the value of
% Player 2's hand after SC and E() denotes the conditional expected value. Each game consists of 10
% hands. Report the mean difference in earnings between Player 1 and Player 2. What can you
% conclude?