| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Computer science is fundamentally problem-solving. We can think of problem-solving as the process of taking some input (details about our problem) and generate some output (the solution to our problem). The "black box" in the middle is computer science.
Vectors must contain the same type of data, that is the entries must all be of the same type such as: character, logical (TRUE or FALSE), or numeric. You can add values to a vector.
L <- c("Chris Saden", "Lauren Castellano","Sarah Spikes","Dean Eckles","Andy Brown", "Moira Burke","Kunal Chawla")
V <- c(100,200,300)
V <- seq(from=100, to=300, by=100)
N <- c(1:10); N
N <- c(N, 11:20); N
[1] 1 2 3 4 5 6 7 8 9 10
[1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
names(V) <- c(‘human’, ‘animal’, ‘tree’); V
mystery = nchar(L); mystery
[1] 11 17 12 11 10 11 12
mystery == 11 L[mystery == 11]
[1] TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE
[1] "Chris Saden" "Dean Eckles" "Moira Burke"
A <- matrix(c(1,2,3,4,5,6), 2,3); A
b_num <- 1:6 b_dim <- c(2,3) B <- matrix(b_num, b_dim); B
C <- matrix(nrow=2,ncol=3) C[,1] <- c(1,2) C[,2] <- c(3,4) C[,3] <- c(5,6); C
Q1. Generate 20 random samples from Poisson with (lambda:2.5) and creating matrix with nrow=4, thus ncol=5 ?
P <- matrix(rpois(20, 2.5), nrow=4); P qplot(rpois(20, 2.5), binwidth=1)
Q2. Generate 20 random samples from Exponential with (lambda:2.5), creating matrix with nrow=4, ncol=5, then generate 20 random samples from "CDF of Exponential" with (lambda:2.5), creating matrix with nrow=4, thus ncol=5 ? Write a sample generater in the 'inv_exp' function.
E <- matrix(rexp(20, 2.5), nrow=4); E
n <- 20
lambda <- 2.5
u <- runif(n)
x <- - log(1-u)/lambda
F <- matrix(x, nrow=4); F
qplot(x)
inv_exp <- function(n, lambda) {
u <- runif(n)
x <- - log(1-u)/lambda
x
}
qplot(inv_exp(20, 2.5))
A <- matrix(c(1,2,3,4,5,6), 2,3); A b_num <- 1:6 b_dim <- c(2,3) B <- matrix(b_num, b_dim); B
rowSums(A) colSums(A) apply(A, 1, sum) #1 is row apply(A, 2, sum) #2 is col
A <- cbind(A, apply(A, 1, sum)) A <- rbind(A, apply(A, 2, sum)); A B <- cbind(B, rowSums(B)) B <- rbind(B, colSums(B)); B
colnames(A) <- c(1:3, 'SUM') rownames(A) <- c(1:2, 'SUM'); A
r_col <- numeric(740) p_col <- numeric(740) obv <- target test_matrix <- cbind(obv,r_col,p_col); head(test_matrix)
density <- numeric(740) test_matrix <- cbind(test_matrix, density); head(test_matrix)
test_matrix[, !colnames(test_matrix) %in% c("density")]; head(test_matrix)
test_matrix <- test_matrix[ , -4]; head(test_matrix)
test_df <- as.data.frame(test_matrix)
x1 <- c(8,2,6,4,5,3,7,1,9,10)
x2 <- c('d','i','i','i','e','i','g','i','i','c')
x3 <- c('TRUE','TRUE','FALSE','NA','FALSE','TRUE','NA','TRUE','FALSE','FALSE')
is.na(x3) <- x3=='NA' #'creating missing data..
X <- data.frame(x1, x2, x3); X
names(X) dim(X) str(X) summary(X) is.na(x3) sum(is.na(X)) apply(X, 2, is.na) apply(apply(X, 2, is.na), 2, sum) #delete all rows with 'na' Y = na.omit(X); Y
Want to drop some ?
X$x3[4] <- X$x3[7] <- NULL doesnt work.. WHY? it only works when it's an "entire column"..for example,
X$x3 <- NULL #:it works.. X[3, ] <- NULL #:it doesn't work..'NULL' only works with columns? X <- X[-3] #:it drops a column X <- X[-3, ] #:it drops a row X <- X[-3:-5, ] #:it drops rows.....Use'-'when dropping rows..
X[ ,c(1,3)] X[3:5, c(1,3)] X[3,1:2] X[sample(1:8, 4), ] #in all rows, choose 4 rows at random.. X[order(x1), ] #sort by row based on column 'x1' X[X$x1 > 5 & X$x2 == 'i', ] #what the fuck is the which?? X[which(X$x1 > 5 | X$x2 == 'i'), ] X[ ,sapply(X, is.numeric)] #show all columns..that consist of numbers..WHY sapply()? X[-c(2,5), ] #show all rows except the row 2, 5 X[!(x2=='i'), ] #show all rows except x2 are 'i'
table(x1) table(x2) table(x3) table(x1, x2) table(x1,x2,x3)
poisson generates integers..then we can convert the table-result into a dataframe!
y <- rpois(200, 1.5); y table(y) data.frame(table(y))
A <- rep(c(1:3), each = 3)
A1 <- rep(c(1:3), 3)
df <- data.frame(A,A1); df
x <- runif(10); x #random uniform
y <- letters[5:14]; y
z <- sample(c(rep('T',5), rep('F',5))); z #random T/F
yxz <- data.frame(y,x,z); yxz
subset(mtcars, mpg>=30|hp<60, c(vs,am)) mtcars[mtcars$mpg>=30|mtcars$hp<60, ] # the above is better than this.
factor() Factor variables are categorical variables that can be either numeric or string. There are a number of advantages to converting categorical variables to factor variables. First, they can be used in statistical modeling where they will be implemented correctly, i.e., they will then be assigned the correct number of degrees of freedom. Factor variables are also very useful in many different types of graphics. Furthermore, storing string variables as factor variables is a more efficient use of memory. The only required argument is a vector of values which can be either string or numeric.
'levels' argument: determines the categories of the factor variable, and the default is the sorted list of all distinct values.
'labels' argument: a vector of values that will be the labels of the categories in the levels argument.
'exclude' argument: defines which levels will be classified as NA in any output using the factor variable.
The first factor 'a_f' sample() Generate a vector of binary data with the size of 20
a <- sample(0:1, 20, replace = TRUE); a is.factor(a) is.numeric(a)
factor(data, labels) The first label, private, will correspond to a=0 and the second label, public, will correspond to a=1 because the order of the labels will follow the numeric order of the data.
a_f <- factor(a, labels=c('private','public')); a_f
is.factor(a_f)
ses <- c("low", "middle", "low", "low", "low", "low", "middle", "low", "middle",
"middle", "middle", "middle", "middle", "high", "high", "low", "middle",
"middle", "low", "high")
is.character(ses)
[1] TRUE
Creating another factor variable called 'ses_f'
ses_f <- factor(ses) is.factor(ses_f) levels(ses_f)
[1] "high" "low" "middle"
The problem here is that the levels are ordered according to the alphabetical order of the categories of 'ses'.
factor(data, levels) In order to fix the ordering we need to use the levels argument to indicate the "correct ordering" of the categories.
ses.f <- factor(ses, levels = c("low", "middle", "high"))
is.factor(ses.f)
levels(ses_f)
[1] "high" "low" "middle"
ordered(levels) We can also create 'ordered factor'. This function has the same arguments as the factor().
ses_order <- ordered(ses, levels=c("low", "middle", "high"))
is.factor(ses_order)
levels(ses_order)
[1] "low" "middle" "high"
Let's add a level !
ses_f[21] <- 'veryhigh' ses_f
[1] low middle low low low low middle low middle middle middle middle middle high high low middle middle low high [NA]
We can see that instead of changing from “high” to “very.high”, the label was changed from “high” to [NA].
To do this correctly, we need to first add the new level, “very.high”, to the factor variable 'ses.f' which we do by using the factor function with the levels argument. Then we can finally add an element to the factor variable from the new level.
ses_f <- factor(ses_f, levels = c(levels(ses_f), "veryhigh")) ses_f[21] <- "veryhigh" ses_f
[1] low middle low low low low middle low middle middle middle middle middle high high low middle middle low high veryhigh
Let's drop the level. Note! you cannot directly remove the factor, so need to create new variable 'ses_ff' and assign factor again.
ses_ff <- ses_f[ses_f != "veryhigh"]; ses_ff ses_ff <- factor(ses_ff) levels(ses_ff)
[1] "high" "low" "middle"
Create an additional continuous variable called 'read' with a size 0f 20 which contains the reading scores.
read <- c(34, 39, 63, 44, 47, 47, 57, 39, 48, 47, 34, 37, 47, 47, 39, 47, 47, 50, 28, 60)
Combining all the variables in a data frame
combo <- data.frame(a, a_f, ses, ses_f, read); combo table(a, ses) table(a_f, ses_f)
bwplot() As in the tables the factor variable will indicate a better ordering of the graphs as well as add useful labels.
library(lattice) bwplot(a_f ~ read | ses_ff, data = combo, layout = c(2, 2))
Datasets should be manipulated into a specified format. Not all datasets look nice. Sometimes we pull data from different sources such as - webpage, pdf, etc. We may need to reshape or rearrange our data into different format. This is a necessary step prior to conducting DataAnalysis. Load your data and get it in the right format before you can visualize it. #There are also packages to help with formatting, mainly 'dplyr' and 'tidyr'.
1. Load files
income <- read.csv("C:/Users/Minkun/Desktop/classes_1/NanoDeg/1.Data_AN/L7/---New R/data/ACS_13_5YR_S1903/ACS_13_5YR_S1903.csv")
head(income, 2)
str(income)
Sometimes, Be more specific. R treated the column of data as numeric instead of a character, but 'GEO.id2' is str. You can specify this with the 'colClasses' argument. Note:'stringsAsFactors' is set to FALSE. Just do this always.
income <- read.csv("C:/~/ACS_13_5YR_S1903/ACS_13_5YR_S1903.csv", stringsAsFactors=FALSE, sep=",", colClasses=c("GEO.id2"="character"))
income <- read.csv("C:/~/ACS_13_5YR_S1903/ACS_13_5YR_S1903.tsv", stringsAsFactors=FALSE, sep="\t", colClasses=c("GEO.id2"="character"))
income <- read.csv("http://datasets.flowingdata.com/tuts/2015/load-data/ACS_13_5YR_S1903.csv", stringsAsFactors=FALSE, sep=",", colClasses=c("GEO.id2"="character"))
income[1:5,c(1,2)]
2. Subset
Maybe you just want to look at income estimates for the total population for now. This is the first seven columns (with the first two indicating region)
income_total <- income[,1:7] head(income_total, 3) summary(income_total)
You can also subset based on values using subset(). Let's say, maybe you only want to look at states in the 'upper quartile'.
income_upper <- subset(income_total, HC02_EST_VC02 >= 58985)
You can also remove rows with missing values in any of the fields.
income_wo_na <- na.omit(income); income_wo_na income_wo_na <- na.omit(income_total); income_wo_na
In this case, you get an empty data frame, because every state has at least one missing value amongst the 153 fields. In contrast, if you ran the function with income_total, you’d just get the same data frame, because no values are missing for the first seven columns.
3. Edit
Change column names
names(income_total) <- c("id", "FIPS", "name", "households", "households_moe", "med_income", "med_income_moe")
In income_total, you have median and margin of error(moe). Maybe you want to add a column for minimum and another for maximum. That’s the median income column plus and minus the margin of error.
income_total$med_min <- income_total$med_income - income_total$med_income_moe income_total$med_max <- income_total$med_income + income_total$med_income_moe
Convert existing column (units to the thousands). When you divide a vector by a single number, the operation is performed on each element of that vector. Same with other basic operations.
income_total$med_min <- income_total$med_min / 1000 income_total$med_max <- income_total$med_max / 1000
4. Merge
Load the datasets.
income2008 <- read.csv("C:/Users/Minkun/Desktop/classes_1/NanoDeg/1.Data_AN/L7/---New R/data/ACS_08_3YR_S1903/ACS_08_3YR_S1903.csv", stringsAsFactors=FALSE, sep=",", colClasses=c("GEO.id2"="character"))
income2013 <- read.csv("C:/Users/Minkun/Desktop/classes_1/NanoDeg/1.Data_AN/L7/---New R/data/ACS_13_5YR_S1903/ACS_13_5YR_S1903.csv", stringsAsFactors=FALSE, sep=",", colClasses=c("GEO.id2"="character"))
Subset
income2008p <- income2008[,c("GEO.id2", "HC02_EST_VC02", "HC02_MOE_VC02")]
income2013p <- income2013[,c("GEO.id2", "HC02_EST_VC02", "HC02_MOE_VC02")]
Rename headers. For the id, which is actually a FIPS (Federal Information Processing Standards) code, we make it the same in both datasets.
names(income2008p) <- c("FIPS", "med2008", "moe2008")
names(income2013p) <- c("FIPS", "med2013", "moe2013")
Combine and bring this together
income0813 <- merge(income2008p, income2013p, by="FIPS") head(income0813, 3)
Save these two data frames as CSV files to use later on in a different program. Specify the dataframe, file destination, and the separator. Row names are used by default, but I typically set it to "FALSE".
write.table(income_total, "C:/Users/Minkun/Desktop/classes_1/NanoDeg/1.Data_AN/L7/---New R/data/income-totals.csv", row.names=FALSE, sep=",") write.table(income0813, "C:/Users/Minkun/Desktop/classes_1/NanoDeg/1.Data_AN/L7/---New R/data/income-2008-13.csv", row.names=FALSE, sep=",")
STR
LIST
LOOP
colors = ["red", "green", "blue", "purple"] for color in colors: print(color)
colors = ["red", "green", "blue", "purple"]
for i in range(len(colors)):
print(colors[i])
---------------------------------------------------------------------------
presidents = ["Washington", "Adams", "Jefferson", "Madison", "Monroe", "Adams", "Jackson"]
for i in range(len(presidents)):
print("President {}: {}".format(i + 1, presidents[i]))
colors = ["red", "green", "blue", "purple"] i = 0 while i < len(colors): print(colors[i]) i += 1
presidents = ["Washington", "Adams", "Jefferson", "Madison", "Monroe", "Adams", "Jackson"]
for num, name in enumerate(presidents, start=1):
print("President {}: {}".format(num, name))
colors = ["red", "green", "blue", "purple"]
ratios = [0.2, 0.3, 0.1, 0.4]
for i, color in enumerate(colors):
ratio = ratios[i]
print("{}% {}".format(ratio * 100, color))
---------------------------------------------------------------------------
colors = ["red", "green", "blue", "purple"]
ratios = [0.2, 0.3, 0.1, 0.4]
for color, ratio in zip(colors, ratios):
print("{}% {}".format(ratio * 100, color))
Select from a dataframe
https://pandas.pydata.org/pandas-docs/stable/merging.html
Write a function that returns the most frequently occurring word in a given string. For example, if the input s is s = 'I scream you scream we all scream for ice cream', the result should be scream as it is the most frequent word in this sentence. If there is a tie for the most common word, return only one result, the first (tied) word in alphabetical order. # HINT: Use the built-in split() function to transform the string s into an array # HINT: Iterate through the array and count each occurance of every word using the .count() list method # HINT: Find the number of times the most common word appears using max() # HINT: Locate the index of the most frequently seen word # HINT: Return the most frequent word. Remember that if there is a tie, return the first (tied) word in alphabetical order.
def most_frequent(s):
ar = s.split()
pool = []
for i in range(0,len(ar),1):
cc = ar.count(ar[i]) ## it shows how many times the element appears in the str ##
pool.append(cc)
mx = max(pool) ## it shows the biggest element. ##
ind_loc = pool.index(mx) ## it gives the location of the element. ##
result = ar[ind_loc]
return result
def test_run():
print most_frequent("cat bat mat cat bat cat") ## output: 'cat' ##
print most_frequent("betty bought a bit of butter but the butter was bitter") ## output: 'butter' ##
print most_frequent("london bridge is falling down falling down falling down london bridge is falling down my fair lady")
if __name__ == '__main__':
test_run()
import datetime my_day = datetime.datetime(1981, 8, 10, 21, 30, 45); my_day
/// datetime.datetime(1981, 8, 10, 21, 30, 45) --- This is a 'datetime object!!!
print(my_day)
/// 1981-08-10 21:30:45
my_day.strftime('%Y-%m-%d')
/// '1981-08-10' --- I converted it to str!!!
today = datetime.datetime.today() # or datetime.datetime.now() print(today)
/// 2018-01-23 22:50:03.543023 --- This is a 'datetime object!!!
year = datetime.datetime.today().year month = datetime.datetime.today().month day = datetime.datetime.today().day hour = datetime.datetime.today().hour minute = datetime.datetime.today().minute second = datetime.datetime.today().second
date_c = datetime.date(2016, 12, 24) # introducing certain day #
date_d = datetime.date(year, month, day) # today's date #
date_e = date_d - date_c ### calculating days passed. ###
print('First meetup : {}'.format(date_c))
print('Today is : {}'.format(date_d))
print("It's been {}days".format(date_e.days))
time_a = datetime.time(21, 30, 24)
print('we meet at: {}'.format(time_a))
today_2 = datetime.datetime.today().strftime('%Y + %m + %d %H peekaboo %M:%S')
print(today_2)
type(today_2)
d1 = datetime.datetime.strptime('2017/12/21 14|00|38', '%Y/%m/%d %H|%M|%S')
d2 = d1.replace(year=9999, month=9, day=29, hour=11, minute=59, second=49)
print('Check-in :', end=' ')
print(d1)
print('New Ckeck-in :', end=' ')
print(d2)
date_a = "2016-07-17 12:23:45" date_b = "Sun, 17 Jul 2016 12:34:56 GMT" d3 = datetime.datetime.strptime(date_a, "%Y-%m-%d %H:%M:%S") d4 = datetime.datetime.strptime(date_b, "%a, %d %b %Y %H:%M:%S GMT") print(d3) print(d4) d3.timestamp(), d4.timestamp()
a = datetime.datetime.fromtimestamp(d3.timestamp()) print(a)
b = datetime.datetime.combine(datetime.date.today(), datetime.time(22,22,22)) print(b)
# 3 days later print(datetime.date(2018,2,4) + datetime.timedelta(days=3)) print(datetime.datetime(2018,2,4,5,36,35) + datetime.timedelta(days=3)) # 7 hours later print(datetime.datetime(2018,2,4,5,36,35) + datetime.timedelta(hours=7)) # etc print(datetime.datetime(2018,2,4,5,36,35) + datetime.timedelta(weeks=2,days=3,hours=-3,minutes=30)) print(datetime.datetime(2018,2,4,5,36,35) + datetime.timedelta(minutes=3,milliseconds=-20,microseconds=400))
time module
import time tt = time.gmtime() print(t2.tm_year) print(t2.tm_mon) print(t2.tm_mday) ## print(t2.tm_hour) print(t2.tm_min) print(t2.tm_sec) print(t2.tm_wday) ## print(t2.tm_yday) ## print(t2.tm_isdst) ##?????
365 - tt.tm_yday
timestr=time.ctime(1234567890) #'Sat Feb 14 08:31:30 2009' is 1234567890 sec time.strptime(timestr, "%a %b %d %H:%M:%S %Y")
#35 days in sec (day-hour-min-sec) = 3024000 35 * 24 * 60 * 60 #3 weeks in sec (week-day-hour-min-sec) = 1814400 3 * 7 * 24 * 60 * 60 #1 week in sec (week-day-hour-min-sec) = 604800 1 * 7 * 24 * 60 * 60
Write a function that returns 'the day' that follows a specified time period after an initial date. Time periods can be specified in two different ways: as a number of days like "1 day" or "30 days", or as a number of weeks like "2 weeks" or "12 weeks". This function takes as input a string depicting a date in YYYY/mm/dd format and a string stating a time period in the form of "X day(s)" or "Y week(s)". Output should be a string in form YYYY/mm/dd with the date that is X days or Y weeks after the initial date.
ef which_date(start_date,t):
import datetime
initial = datetime.date(int(start_date.split('/')[0]), int(start_date.split('/')[1]), int(start_date.split('/')[2]))
if 'day' in t:
nex = datetime.timedelta(int(t.split()[0]))
elif 'week' in t:
nex = datetime.timedelta(int(t.split()[0])*7)
end_date = (initial + nex).strftime('%Y/%m/%d')
return end_date
def test():
assert which_date('2016/02/10','35 days') == '2016/03/16'
assert which_date('2016/12/21','3 weeks') == '2017/01/11'
assert which_date('2015/01/17','1 week') == '2015/01/24'
print("All tests completed.")
if __name__ == "__main__":
test()
(https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html)
class = blue-print
Each class is a blue-print describing the attributes and functionality of an object or abstract concept (For example, houses, bank accounts, employees, clients, cars, products...)
Class names are nouns and they should start with an uppercase letter. If the name has more than one word, each word should be capitalized.
In short, a class contains the data, and the action of the objects that belong to a class.
Main Elements
class <ClassName>:
# "Class" Attributes
<class attribute> = <value>
# "Constructor" and "Instance" Attributes
def __init__(self, <parameters>):
self.<instance attribute1> = <value1>
self.<instance attribute2> = <value2>
. . .
# Methods
def <method_name> (self, <parameters>):
. . .
The instance attribute is defined within the constructor in the class.
instance = object
Instances are created from a class. They are concrete objects created based on the "blue-print"(class), and each of which is totally independent object. "class" is like a catagory. If "class" is an abstract representation of a certain object, while "instance" is a concrete example of a certain object develped from the "class".
All objects generated by the same class share the same instance attribute, but the values of the instance attributes for each object are independent!

1> Let's define the instance attributes in the class. First, think about what features are supposed to vary by each object(instance). If the category is about 'dog', then 'name' might vary! Ok! then build the instance attribute of 'name' inside of the "Constructor". The constructor initializes the instance attributes.
class Dog:
def __init__(self, age_param, name_param, is_male_param, weight_param, ailment_param=None, num_baby_param=0):
self.age = age_param
self.name = name_param
self.is_male = is_male_param # Boolean: True is Male, False if Female.
self.weight = weight_param
self.ailment = ailment_param # String
self.num_baby = num_baby_param # Integer
self.owner = "Minkun~always" # fixed attribute !!! Unnecessary to specify in the argument of the constructor
2> Let's create the instances: <ins_variable_name> = <class_name>(param, param,...)
my_dog = Dog(5, "Selly", True, 15, ["cough", "insomnia"]) # this omitting default implies the dog doesn't have a baby
3> Let's access the instance attributes: <ins_variable_name>.<instance_attribute>
my_dog.name
4> Let's modify the instance attributes(like..updating the value): <ins_variable_name>.<instance_attribute> = <new_value>
my_dog.is_male = False
: It refers to the bundling of Data(attribute) and Actions(method) into a class. It has a purpose of "information hiding" or "access restriction".
: The interface should be independent of the implementation. Let's say the classes should interact with each other as the program grows. We have to decide what part of our code(data? action?) is going to be public(accessible from anywhere) and which part (data? action?) is only accessible within a class. "Interface" as a USER'S VIEW is a part of the code that is publicly available for use, and "Implementation" as a DEVELOPER'S VIEW is a part of the code that relates "how the code is written to achieve the goal functionality"?

Let's see how attributes works to achieve the principles of Encapsulation & Abstraction! Use Privacy / Publicity ?
class Employee:
def __init__(self, name_param, age_param, address_param, number_param, vehicle_param=None):
self.name = name_param
self._age = age_param # "_"<protection-01>: Not To Access this attribute from outside of the class...
self._address = address_param # "_"<protection-01>: Not To Access this attribute from outside of the class...
self.__number = number_param # "__"<protection-02>: Not To Access this attribute from outside of the class...
self.__vehicle = vehicle_param # "__"<protection-02>: Not To Access this attribute from outside of the class...
employee01 = Employee("John", 56, "5th Ave", 34523, "Honda")
employee01._age # it works
employee01.__number # it won't work coz it's in a "name mangling" process.
employee01._Employee.__number # it works if it comes with its class name.
Getters and Setters are actions as members of a class...so they are "Methods" for the instance that can call them. Their purpose is to "get" and "set" the value of an instance attribute. They protect data by providing an indirect way to access.
| Back | FazBrowse Home | New Git URL |