Action Rules Discovery without pre-existing classification rules By

Action Rules Discovery without pre-existing classification rules By Zbigniew W. Ras 1, 2 and Agnieszka Dardzinska 1, 3 1) 2) 3) University of North Carolina, Charlotte, NC, USA Polish Academy of Science, Warsaw, Poland Bialystok Technical University, Bialystok, Poland

1. Introduction : Action Rule An action rule is a rule extracted from a datatable that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Information System

1. Introduction : Action Rule An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Assumption: Information System - attributes are partitioned into stable and flexible

1. Introduction : Action Rule An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Assumption: Information System - attributes are partitioned into stable and flexible Values of attributes can be changed

1. Introduction : Action Rule Action rule is defined as a term [(ω) ∧ (α → β)] →(ϕ→ψ) conjunction of fixed condition features shared by both groups Information System proposed changes in values of flexible features desired effect of the action

2. Background and Objectives Information System S Information System X a b c d x 1 a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 S = (X, A, V) X = {x 1 , x 2 , x 3 , x 4 , x 5 , x 6 , x 7 , x 8 } A = {a, b, c, d} X is a set of objects A is a set of attributes (partial functions: X V) Va is a set of values of attribute a, where a A, and V = {Va : a A},

3. ARED Algorithm (Stable and Flexible Att) Decision System X c d a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 Constraints: Minimum Support 1 Minimum Confidence 2 b x 1 Decision Attribute : {d} Stable Attributes : Ast= {a} Flexible Attributes : Afl = {b, c} a a 2 b 3 c 2 d 1 x 6 a 1 b 1 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 d 2

2. Background and Objectives Action rule generation in earlier work used existing classification rules (generated, for instance, by rule discovery algorithms such as LERS or ERID) Proposed Algorithm 1. Extract action rules directly from an information system without using pre-existing classification rules. 2. Extract action rules that have minimal attribute involvement.

Action Rules Discovery System DEAR Objects x 1, x 2, x 3, x 4 a b c d 0 x 1, x 3 0 x 2, x 4 L 2 x 2, x 4 x 7, x 8 2 x 7, x 8 1 Decision Attribute: d L 2 H b Objects x 7, x 8 a b 1 c x 5, x 6 2 c=? a=? b c 1 Objects b c=? Objects c 1 x 7, x 8 3 2 x 2, x 4 c=2 Objects b x 7, x 8 1 T 1 x 7, x 8 1 x 5, x 6 T 2 b 3 T 4 c c=? c=1 Objects x 2, x 4 x 1, x 3 b x 1, x 2, x 3, x 4 3 Objects 2 Objects 1 c=0 b a=0 2 x 2, x 4 1 a=2 1 0 x 2, x 4 2 x 7, x 8 2 c x 1, x 3 x 7, x 8 0 a=? Objects c 0 x 5, x 6 Set of rules R with supporting objects Objects b x 2, x 4 H 1 a x 1, x 3 x 2, x 4 Flexible Attribute: b L 3 x 1, x 2, x 3, x 4 Stable Attributes: {a, c} L 1 x 5, x 6 Objects L Objects x 1, x 2, x 3, x 4 b T 6 b T 5 (T 3, T 1) : (a = 2) (b, 2 1) ( d, L H) (a = 2) (b, 3 1) ( d, L H)

3. Action Rules Atomic Action Term: (a, a 1 → a 2 ) a 1, a 2 ∈ Va attribute If a 1 = a 2 then a – stable on a 1

3. Action Rules Atomic Action Term: (a, a 1 → a 2 ) a 1, a 2 ∈ Va attribute If a 1 = a 2 then a – stable on a 1 Set of Action Terms: 1. If t – atomic action term, then t – an action term 2. If t 1, t 2 – action terms, then t 1*t 2 – an action term 3. If t – action term containing sub-terms (a, a 1 → a 2 ), (b, b 1 → b 2), then a ≠ b Domain of Action Term t: Dom(t) – set of all attribute names listed in t

3. Action Rules Atomic Action Term: (a, a 1 → a 2 ) a 1, a 2 ∈ Va attribute If a 1 = a 2 then a – stable on a 1 Action Rule: r = [t 1 → t 2] action term Dom(t 2)={d} and Dom(t 1) in A atomic action term

3. Example Decision System S X a b c d x 1 a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 (a, a 2 → a 2) (b, b 1 → b 3) (c, c 2 → c 2) (d, d 1 → d 2) atomic action terms r=[[(a, a 2→a 2)*(b, b 1→b 3)] → (d, d 1→d 2) Dom (r) ={a, b, d} action rule

4. Interpretation NS of action terms Information System S=(X, A, V) 1. If (a, a 1 → a 2) – atomic action term then: NS((a, a 1 → a 2))=[{x∈X: a(x)=a 1}, {x∈X: a(x)=a 2}] 2. If t 1=(a, a 1 → a 2) * t and NS(t)=[Y 1, Y 2], then: NS(t 1)=[ Y 1 ∩ {x∈X: a(x)=a 1}, Y 2 ∩ {x∈X: a(x)=a 2} ] Assume that [Y 1, Y 2]∩[Z 1, Z 2] = [Y 1∩Z 1, Y 2∩Z 2], and NS(t 1)=[Y 1, Y 2], NS(t 2)=[Z 1, Z 2], then: NS(t 1*t 2)=NS(t 1)∩NS(t 2)

4. Interpretation NS of action terms Information System S=(X, A, V) NS(t 1)=[Y 1, Y 2] NS(t 2)=[Z 1, Z 2] Action Rule: Support: Confidence: r = [t 1 → t 2]

Information System S X a b c d x 1 a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 r = [[(a, a 2→a 2)*(b, b 1→b 2)] → (d, d 1→d 2) NS((a, a 2→a 2))=[{x 2, x 3, x 4, x 5}, {x 2, x 3, x 4, x 5}] NS((b, b 1→b 2))=[{x 1, x 2, x 4, x 6}, {x 3, x 7, x 8}] NS((d, d 1→d 2))=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}]

Information System S X a b c d x 1 a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 r = [[(a, a 2→a 2)*(b, b 1→b 2)] → (d, d 1→d 2) d 3 NS((a, a 2→a 2))=[{x 2, x 3, x 4, x 5}, {x 2, x 3, x 4, x 5}] NS((b, b 1→b 2))=[{x 1, x 2, x 4, x 6}, {x 3, x 7, x 8}] NS((d, d 1→d 2))=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] NS((a, a 2→a 2)*(b, b 1→b 2))=[{x 2, x 4}, {x 3}] sup (r) = 2, conf (r) = 0

5. Algorithm ARED -Similar to ERID and LERS -Only positive marks yield action rules -Action terms of length k are built from unmarked action terms of length k – 1 and unmarked atomic action terms of length 1 S=(X, A ∪ {d}, V) – decision system λ 1, λ 2 – minimum support and confidence CS(a)={NS(ta): ta – an atomic action term built from elements in Va} a∈A

5. ARED Algorithm Object reclassification from class d 1 to d 2 X a b c d x 1 a 1 b 1 c 1 d 1 x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 For decision attribute in S: NS(d, d 1 d 2)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] Notation: t 1=(b, b 1), t 2=(b, b 2), t 3=(b, b 3), t 4=(a, a 1 a 2), t 5=(a, a 1), t 6=(a, a 2), t 7=(a, a 2 a 1), t 8=(c, c 1 c 2), t 9=(c, c 2 c 1), t 10=(c, c 1), t 11=(c, c 2), t 12 = (d, d 1 d 2), stable attribute flexible attributes λ 1=2, λ 2=1/4

5. ARED Algorithm Object reclassification from class d 1 to d 2 λ 1=2, λ 2=1/4 For decision attribute in S: NS(d, d 1 d 2)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: Not marked λ 1=3 NS(t 1)=[{x 1, x 2, x 4, x 6}, {x 1, x 2, x 4, x 6}] Mark “-” λ 1=1 NS(t 2)=[{x 3, x 7, x 8}, {x 3, x 7, x 8}] Mark “-” λ 1=1 NS(t 3)=[{x 5}, {x 5}] Mark “-” λ 2=0 NS(t 4)=[{x 1, x 6, x 7, x 8}, {x 2, x 3, x 4, x 5}] Not marked λ 1=4 NS(t 5)=[{x 1, x 6, x 7, x 8}, {x 1, x 6, x 7, x 8}] Mark “-” λ 2=0 NS(t 6)=[{x 2, x 3, x 4, x 5}, {x 2, x 3, x 4, x 5}] Mark “+” λ 1=4, λ 2=1/4 NS(t 7)=[{x 2, x 3, x 4, x 5}, {x 1, x 6, x 7, x 8}]

5. ARED Algorithm Object reclassification from class d 1 to d 2 λ 1=2, λ 2=1/4 For decision attribute in S: NS(t 12)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: Not marked λ 1=3 NS(t 1)=[{x 1, x 2, x 4, x 6}, {x 1, x 2, x 4, x 6}] Marked “-” λ 2=0 NS(t 2)=[{x 3, x 7, x 8}, {x 3, x 7, x 8}] Marked “-” λ 1=1 NS(t 3)=[{x 5}, {x 5}] Marked “-” λ 2=0 NS(t 4)=[{x 1, x 6, x 7, x 8}, {x 2, x 3, x 4, x 5}] Not marked λ 2=2 NS(t 5)=[{x 1, x 6, x 7, x 8}, {x 1, x 6, x 7, x 8}] Marked “-” λ 2=0 NS(t 6)=[{x 2, x 3, x 4, x 5}, {x 2, x 3, x 4, x 5}] Mark “+” λ 1=4, λ 2=1/4 NS(t 7)=[{x 2, x 3, x 4, x 5}, {x 1, x 6, x 7, x 8}] r = [t 7 t 1]

5. ARED Algorithm Object reclassification from class d 1 to d 2 λ 1=2, λ 2=1/4 For decision attribute in S: NS(t 12)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: Not marked NS(t 8)=[{x 1, x 4, x 8}, {x 2, x 3, x 5, x 6, x 7}] Rule r = [t 8→t 12], conf = 2/3 *1/5 <λ 2, sup=2 ≥ λ 1 Marked “-” NS(t 9)=[{x 2, x 3, x 5, x 6, x 7}, {x 1, x 4, x 8}] Marked “-” NS(t 10)=[{x 1, x 4, x 8}, {x 1, x 4, x 8}] Not marked NS(t 11)=[{x 2, x 3, x 5, x 6, x 7}, {x 2, x 3, x 5, x 6, x 7}]

5. ARED Algorithm Object reclassification from class d 1 to d 2 Now action terms of length 2 from unmarked action terms of length 1 λ 1=2, λ 2=1/4 For decision attribute in S: NS(t 12)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: Not marked NS(t 1*t 5)=[{x 1, x 6}, {x 1, x 6}] Marked “+” NS(t 1*t 8)=[{x 1, x 4}, {x 2, x 6}] Rule r = [t 1*t 8→t 12], conf = 1/2 ≥ λ 2, sup=2 ≥ λ 1 Not marked NS(t 1*t 11)=[{x 2, x 6}, {x 2, x 6}] Marked “-” NS(t 5*t 8)=[{x 1, x 8}, {x 6, x 7}] Rule r = [t 5*t 8→t 12], conf = 1/2 ≥ λ 2, sup= 1 < λ 1 Not marked NS(t 5*t 11)=[{x 6, x 7}, {x 6, x 7}] Marked “-” NS(t 8*t 11)=[Ø, {x 2, x 3, x 5, x 6, x 7}]

5. ARED Algorithm Object reclassification from class d 1 to d 2 Now action terms of length 3 from unmarked action terms of length 2 λ 1=2, λ 2=1/4 For decision attribute in S: NS(t 12)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: (t 1*t 5*t 8) –extension of (t 5*t 8) Marked “-” Algorithm STOPS!

5. ARED Algorithm Object reclassification from class d 1 to d 2 λ 1=2, λ 2=1/4 For decision attribute in S: NS(t 12)=[{x 1, x 2, x 3, x 4, x 5, x 7}, {x 6}] For classification attribute in S: X a b c d x 1 a 1 b 1 c 1 d 1 [[(b, b 1→b 1)*(c, c 1→c 2)] → (d, d 1→d 2)] x 2 a 2 b 1 c 2 d 1 x 3 a 2 b 2 c 2 d 1 [[(a, a 2→a 1] → (d, d 1→d 2)] x 4 a 2 b 1 c 1 d 1 x 5 a 2 b 3 c 2 d 1 x 6 a 1 b 1 c 2 d 2 x 7 a 1 b 2 c 2 d 1 x 8 a 1 b 2 c 1 d 3 Action rules:

6. Application Database HEPAR (Medical Center of Postgraduate Education, Warsaw, Poland) prepared by Dr. med. Hanna Wasyluk § 758 patients described by 106 attributes (including 31 laboratory tests with values discretized to: “below normal”, “above normal”). 14 attributes are stable. Two tests are invasive tests: HBs. Ag [in tissue], tests: HBc. Ag [in tissue]. There are 7 decision values: I. Acute hepatitis IIa. Subacute hepatitis (types BC) IIa. IIb. Subacute hepatitis (alcohol-abuse) IIb. IIIa. Chronic hepatitis (curable) IIIb. Chronic hepatitis (non-curable) IV. Cirrhosis hepatitis V. Liver cancer We can ask for re-classifications: IIb I, IIIa I

6. Application Database HEPAR (heavily incomplete) Handling data incompleteness § Attributes with more than 90% of null values have been removed § § Subjective attributes have been removed [example: History of alcohol abuse] [example: § Removal of absolute medical tests § RSES null value imputation module was used to make the resulting dataset complete.

6. Application Chosen reduct has no invasive tests, 11% incomplete data {m, n, q, u, y, aa, ah, ai, am, an, aw, bb, bg, by, cj, cm} m n q u y aa ah ai am an aw bb bg bm by cj cm dd Bleeding Subjaundice symptoms Eructation Obstruction Weight loss Smoking History of viral hepatitis (stable) Surgeries in the past (stable) History of hospitalization (stable) Jaundice in pregnancy Erythematous dermatitis Cysts Sharp liver edge (stable) Blood cell plaque Alkaline phosphatase Prothrombin index Total cholesterol Decision attribute

6. Application Action Rules discovery in Database HEPAR § No history of viral hepatitis but with history of surgery and hospitalization, sharp liver edge normal, no subjaundice symptoms, total cholesterol normal, erythematous dermatitis normal, weight normal, no cysts , patient does not smoke § [(ah=1) (ai=2) (am=2) (bg=1)] (cm=1) (aw=1) (u, 1) (bb=1) (aa=1) (q, 1) (m, 1) (n=1) (bm, down) (y=1) (by, norm up) (dd, IIIa I ) § § § [(ah=1) (ai=2) (am=2) (bg=1)] (cm=1) (aw=1) (u, 1) (bb=1) (aa=1) (q, 1) (m, 1) (n=1) (bm, down) (y=1) (by, norm down) (dd, IIIa I ) Two quite similar action rules have been constructed: By getting rid of obstruction, eructation, bleeding, by decreasing the blood cell plaque and by changing the level of alkaline phosphatase we should be able reclassify the patient from IIIa group to I. Medical doctor should decide for a given patient if the alkaline phosphatase level needs to be decreased or increased. Attribute values of total cholesterol, weight, and smoking have to stay unchanged.

7. Conclusion and Future Work • Presented algorithm discovers action rules directly from a decision table. • The proposed algorithm generates the shortest action rules without using pre-existing classification rules • Future work includes - Action rule generation from incomplete information system - Action rule generation when attributes are hierarchical - Considering different flexibilities of attributes (e. g. the social condition is most likely less flexible than the health condition )

Questions? Thank You